Implementation Aspects in Invariance Alignment

Abstract

In social sciences, multiple groups, such as countries, are frequently compared regarding a construct that is assessed using a number of items administered in a questionnaire. The corresponding scale is assessed with a unidimensional factor model involving a latent factor variable. To enable a comparison of the mean and standard deviation of the factor variable across groups, identification constraints on item intercepts and factor loadings must be imposed. Invariance alignment (IA) provides such a group comparison in the presence of partial invariance (i.e., a minority of item intercepts and factor loadings are allowed to differ across groups). IA is a linking procedure that separately fits a factor model in each group in the first step. In the second step, a linking of estimated item intercepts and factor loadings is conducted using a robust loss function $L_{0.5}$. The present article discusses implementation alternatives in IA. It compares the default $L_{0.5}$ loss function with $L_p$ with other values of the power p between 0 and 1. Moreover, the nondifferentiable $L_p$ loss functions are replaced with differentiable approximations in the estimation of IA that depend on a tuning parameter $\mathsf{\epsilon }$ (such as, e.g., $\mathsf{\epsilon }=0.01$). The consequences of choosing different values of $\mathsf{\epsilon }$ are discussed. Moreover, this article proposes the $L_0$ loss function with a differentiable approximation for IA. Finally, it is demonstrated that the default linking function in IA introduces bias in estimated means and standard deviations if there is noninvariance in factor loadings. Therefore, an alternative linking function based on logarithmized factor loadings is examined for estimating factor means and standard deviations. The implementation alternatives are compared through three simulation studies. It turned out that the linking function for factor loadings in IA should be replaced by the alternative involving logarithmized factor loadings. Furthermore, the default $L_{0.5}$ loss function is inferior to the newly proposed $L_0$ loss function regarding the bias and root mean square error of factor means and standard deviations.

1. Introduction

In the comparison of multiple groups in confirmatory factor analysis (CFA), some identifying assumptions have to be made. It is frequently assumed that item parameters are equal across groups, which is denoted as measurement invariance [1]. The invariance concept has been very prominent in psychology and the social sciences in general [2,3]. For example, in international large-scale assessment studies in education, like the programme for international student assessment (PISA), the necessity of invariance is strongly emphasized [4].

In violation of measurement invariance, the invariance alignment (IA) method [5,6] has been proposed to achieve approximate invariance [7]. That is, item parameters should be made as invariant as possible while allowing a few deviations from invariance. By doing so, group comparisons can be made more robust against violations of measurement invariance. Note that IA is also referred to as alignment optimization [8,9].

Although IA can be seen as a canonical method for handling measurement noninvariance in the social sciences [10,11], this method has not been thoroughly studied from a statistical and conceptual point of view. There are a few simulation studies that investigate the behavior of the IA method. Except for a few studies [12,13], all simulation studies were carried out with the popular but commercial (and closed-source) Mplus software [14]. Previous simulation studies for unidimensional factor models investigated the case of continuous items [5,9,15,16,17], dichotomous items [18,19], and polytomous items [13,20]. The extension of IA to multidimensional factor models with continuous items was discussed in [21,22]. IA was studied in longitudinal measurement models in [23,24,25,26]. The IA method has been extended to exploratory structural equation models [22,27]. Moreover, the optimization function used in IA gave rise to extending it to a general framework used in the penalized maximum likelihood estimation of structural equation models [28].

IA has been applied in a wide area of disciplines. For example, IA has been utilized to compare European countries regarding attitudes towards migration [29,30]. Ref. [31] compares seven Latin American countries for the purpose in life test through IA. IA was applied to study bullying for children and adolescents [32,33]. Questionnaire data from the programme for international student assessment (PISA) study [34,35] and the trends in international mathematics and science study (TIMSS) [36] were used in IA applications. Furthermore, the IA method was utilized to investigate overexcitability [37], homophobia [38], distributed leadership [39], and gender role attitudes [40].

In this article, we focus on the implementation aspects of the IA method. The IA optimization function is nondifferentiable. Software implementations of the IA method rely on differentiable approximations that depend on a tuning parameter $\mathsf{\epsilon }$. The default value of this tuning parameter $\mathsf{\epsilon }$ is critically examined in this paper. Furthermore, the originally proposed IA method utilizes the $L_p$ loss function $\mathsf{\rho }\left(x\right)={\left|x\right|}^p$ for $p=0.5$. This article investigates whether other choices than $p=0.5$ result in improved estimation performance of the IA method. Moreover, the IA method uses a particular linking function for determining factor standard deviation based on a quantification of differences in residual item loadings. We show in this article that the performance of IA can be improved by relying on a different linking function that employs logarithmized item loadings to estimate factor standard deviations. Finally, the performance of IA is compared with a recently proposed differentiable approximation of the $L_0$ loss function. It turns out that this loss function performed comparably to default IA implementations, if not better, regarding the root mean square error of parameter estimates.

The rest of this article is organized as follows. IA estimation based on the robust $L_p$ and $L_0$ loss functions is treated in Section 2. Section 3 discusses the standard error computation in IA. In Section 4, the research purpose of this article is outlined. Section 5, Section 6 and Section 7 contain three simulation studies, respectively, that thoroughly investigate the choices in IA implementation. Finally, the article closes with a discussion in Section 8.

2. Loss Functions in Invariance Alignment

In this section, the statistical background of IA is reviewed. In particular, the choice of different loss functions in IA is discussed.

Let $X_{ig}$ denote item i ($i=1,\dots ,I$) in group g ($g=1,\dots ,G$). A unidimensional factor model [41] is defined as

$$X_{ig}={\mathsf{\nu }}_{ig}+{\mathsf{\lambda }}_{ig}{F}_g+{\mathsf{ϵ}}_{ig},F_g\sim \mathsf{N}({\mathsf{\mu }}_g,{\mathsf{\sigma }}_g^2),{\mathsf{ϵ}}_{ig}\sim \mathsf{N}(0,{\omega }_{ig}),$$

(1)

where ${\mathsf{\lambda }}_{ig}$ is an item loading and ${\mathsf{\nu }}_{ig}$ is an item intercept. Without loss of generality, item loadings can be assumed to be positive. The factor variables $F_g$ and all residual variables ${\mathsf{ϵ}}_{ig}$ are independent and normally distributed. The factor variable $F_g$ has a factor mean ${\mathsf{\mu }}_g$ and a factor standard deviation ${\mathsf{\sigma }}_g$.

It must be emphasized that the model parameters in (1) are not identified. An identified model is obtained by assuming a standardized latent variable $F_g$ (i.e., with a mean of 0 and a standard deviation of 1):

$$X_{ig}={\mathsf{\nu }}_{ig,0}+{\mathsf{\lambda }}_{ig,0}{F}_g+{\mathsf{ϵ}}_{ig},F_g\sim \mathsf{N}(0,1),{\mathsf{ϵ}}_{ig}\sim \mathsf{N}(0,{\omega }_{ig}).$$

(2)

The model parameters in (1) and (2) are related to each other by

$${\mathsf{\lambda }}_{ig,0}={\mathsf{\lambda }}_{ig}{\mathsf{\sigma }}_g\mathrm{and}{\mathsf{\nu }}_{ig,0}={\mathsf{\nu }}_{ig}+{\mathsf{\lambda }}_{ig}{\mathsf{\mu }}_g={\mathsf{\nu }}_{ig}+\frac{{\mathsf{\lambda }}_{ig,0}}{{\mathsf{\sigma }}_g}{\mathsf{\mu }}_g.$$

(3)

A convenient property is measurement invariance [1,3], in which the same item loadings and item intercepts across groups can be assumed. That is, there exist item loadings ${\mathsf{\lambda }}_i$ such that ${\mathsf{\lambda }}_i={\mathsf{\lambda }}_{ig}$ for all $g=1,\dots ,G$ and item intercepts ${\mathsf{\nu }}_i$ such that ${\mathsf{\nu }}_i={\mathsf{\nu }}_{ig}$ for $g=1,\dots ,G$ for all items $i=1,\dots ,I$. The absence of measurement invariance is also labeled as differential item functioning (DIF; [2,42]) in the literature. In the case of measurement invariance, Equation (3) can be rewritten as

$${\mathsf{\lambda }}_{ig,0}={\mathsf{\lambda }}_i{\mathsf{\sigma }}_g\mathrm{and}{\mathsf{\nu }}_{ig,0}={\mathsf{\nu }}_i+\frac{{\mathsf{\lambda }}_{ig,0}}{{\mathsf{\sigma }}_g}{\mathsf{\mu }}_g.$$

(4)

The IA method of Asparouhov and Muthén [5,6] tackles situations under sparse violations of measurement invariance. In this case, a few item loadings or item intercepts are allowed to differ across groups, while the majority of items (approximately) fulfill the invariance assumption [43]. This situation is referred to as partial invariance [44].

In IA, the unidimensional factor model (2) is separately estimated for all groups in the first step. The estimated item parameters ${\widehat{\mathsf{\lambda }}}_{ig,0}$ and ${\widehat{\mathsf{\nu }}}_{ig,0}$ ($i=1,\dots ,I$; $g=1,\dots ,G$) are used as the input of the IA. By rewriting (3) and inserting the estimated item loadings and item intercepts, we obtain

$${\mathsf{\lambda }}_{ig}-{\mathsf{\lambda }}_{ih}=\frac{{\widehat{\mathsf{\lambda }}}_{ig,0}}{{\mathsf{\sigma }}_g}-\frac{{\widehat{\mathsf{\lambda }}}_{ih,0}}{{\mathsf{\sigma }}_h}\mathrm{and}{\mathsf{\nu }}_{ig}-{\mathsf{\nu }}_{ih}={\widehat{\mathsf{\nu }}}_{ig,0}-{\widehat{\mathsf{\nu }}}_{ih,0}-\frac{{\widehat{\mathsf{\lambda }}}_{ig,0}}{{\mathsf{\sigma }}_g}{\mathsf{\mu }}_g+\frac{{\widehat{\mathsf{\lambda }}}_{ih,0}}{{\mathsf{\sigma }}_h}{\mathsf{\mu }}_h.$$

(5)

These relations motivate the minimization of the following linking function in IA to determine group means $\mathbf{\mu }=({\mathsf{\mu }}_1,\dots ,{\mathsf{\mu }}_G)$ and standard deviations $\mathbf{\sigma }=({\mathsf{\sigma }}_1,\dots ,{\mathsf{\sigma }}_G)$:

$$H(\mathbf{\mu },\mathbf{\sigma })=\sum _{i=1}^I\sum _{g=1}^{G-1}\sum _{h=g+1}^G{w}_{i1,gh}\mathsf{\rho }\left(\frac{{\widehat{\mathsf{\lambda }}}_{ig,0}}{{\mathsf{\sigma }}_g}-\frac{{\widehat{\mathsf{\lambda }}}_{ih,0}}{{\mathsf{\sigma }}_h}\right)+\sum _{i=1}^I\sum _{g=1}^{G-1}\sum _{h=g+1}^G{w}_{i2,gh}\mathsf{\rho }\left({\widehat{\mathsf{\nu }}}_{ig,0}-{\widehat{\mathsf{\nu }}}_{ih,0}-\frac{{\widehat{\mathsf{\lambda }}}_{ig,0}}{{\mathsf{\sigma }}_g}{\mathsf{\mu }}_g+\frac{{\widehat{\mathsf{\lambda }}}_{ih,0}}{{\mathsf{\sigma }}_h}{\mathsf{\mu }}_h\right),$$

(6)

  where the weights $w_{i1,gh}$ and $w_{i2,gh}$ are known, and $\mathsf{\rho }$ is a loss function. Asparouhov and Muthén [5] proposed $w_{i1,gh}=w_{i2,gh}=\sqrt{n_g{n}_h}$ and $\mathsf{\rho }\left(x\right)=\sqrt{\left|x\right|}$, where $n_g$ denotes the sample size of group g. In the minimization of (6), additional identification constraints must be imposed. In this article, we fix the moments in the first group; that is, we set ${\mathsf{\mu }}_1=0$ and ${\mathsf{\sigma }}_1=1$.

In the rest of the article, we ignore the weights in (6) for the following reasons. First, it simplifies mathematical notation. Second, it is not obvious why one should choose weights related to the sample sizes of the groups. We think that model deviations should be equally weighted across groups.

Note that the optimization function H of IA, defined in (6), can be rewritten as

$$H(\mathbf{\mu },\mathbf{\sigma })=H_1\left(\mathbf{\sigma }\right)+H_2(\mathbf{\mu },\mathbf{\sigma }),\mathrm{where}$$

(7)

$$H_1\left(\mathbf{\sigma }\right)=\sum _{i=1}^I\sum _{g=1}^{G-1}\sum _{h=g+1}^G\mathsf{\rho }\left(\frac{{\widehat{\mathsf{\lambda }}}_{ig,0}}{{\mathsf{\sigma }}_g}-\frac{{\widehat{\mathsf{\lambda }}}_{ih,0}}{{\mathsf{\sigma }}_h}\right)\mathrm{and}{H}_2(\mathbf{\mu },\mathbf{\sigma })=\sum _{i=1}^I\sum _{g=1}^{G-1}\sum _{h=g+1}^G\mathsf{\rho }\left({\widehat{\mathsf{\nu }}}_{ig,0}-{\widehat{\mathsf{\nu }}}_{ih,0}-\frac{{\widehat{\mathsf{\lambda }}}_{ig,0}}{{\mathsf{\sigma }}_g}{\mathsf{\mu }}_g+\frac{{\widehat{\mathsf{\lambda }}}_{ih,0}}{{\mathsf{\sigma }}_h}{\mathsf{\mu }}_h\right).$$

(8)

It has been shown that the simultaneous minimization of H with respect to $\mathbf{\mu }$ and $\mathbf{\sigma }$ can be viewed as a two-step minimization problem [45]. In more detail, a vector of estimated factor standard deviations $\widehat{\mathbf{\sigma }}$ is obtained by minimizing $H_1\left(\mathbf{\sigma }\right)$ in the first step. In the second step, a vector of estimated factor means $\widehat{\mathbf{\mu }}$ is obtained by minimizing $H_2(\mathbf{\mu },\widehat{\mathbf{\sigma }})$ with respect to $\mathbf{\mu }$.

Equation (3) can be rewritten as

$$log{\mathsf{\lambda }}_{ig,0}=log{\mathsf{\lambda }}_{ig}+log{\mathsf{\sigma }}_g\mathrm{and}{\mathsf{\nu }}_{ig,0}={\mathsf{\nu }}_{ig}+\frac{{\mathsf{\lambda }}_{ig,0}}{{\mathsf{\sigma }}_g}{\mathsf{\mu }}_g.$$

(9)

This motivates using an alternative optimization function $H_1^{*}$ for determining standard deviations that employs logarithmized item loadings (see [45,46])

$$H_1^{*}\left({\mathbf{\sigma }}^{*}\right)=\sum _{i=1}^I\sum _{g=1}^{G-1}\sum _{h=g+1}^G\mathsf{\rho }\left(log{\widehat{\mathsf{\lambda }}}_{ig,0}-log{\widehat{\mathsf{\lambda }}}_{ih,0}-{\mathsf{\sigma }}_g^{*}+{\mathsf{\sigma }}_h^{*}\right),$$

(10)

where ${\mathsf{\sigma }}_g^{*}=log{\mathsf{\sigma }}_g$ for $g=1,\dots ,G$. Due to the identification constraint, we fix ${\mathsf{\sigma }}_1^{*}=0$ (i.e., ${\mathsf{\sigma }}_1=exp\left({\mathsf{\sigma }}_1^{*}\right)=1$). By minimizing $H_1^{*}$, a vector of standard deviations ${\widehat{\mathbf{\sigma }}}^{*}$ on the logarithm metric is obtained; that is, ${\widehat{\mathbf{\sigma }}}^{*}=({\widehat{\mathsf{\sigma }}}_1^{*},\dots ,{\widehat{\mathsf{\sigma }}}_G^{*})$. The vector of estimated standard deviations $\widehat{\mathbf{\sigma }}$ can be obtained by exponentiating all entries in ${\widehat{\mathbf{\sigma }}}^{*}$. The vector of estimated factor means $\widehat{\mathsf{\mu }}$ can again be obtained by minimizing $H_2(\mathbf{\mu },\widehat{\mathbf{\sigma }})$.

Hence, there are two estimation options for IA. The original approach of [5] minimizes $H_1$ and is referred to as the “NOL” method (i.e., no logarithm for item loadings). The second approach obtains factor standard deviations by minimizing $H_1^{*}$ and is referred to as the “LOG” method (i.e., taking the logarithmized item loadings for defining deviations).

As mentioned above, IA uses the loss function $\mathsf{\rho }\left(x\right)=\sqrt{\left|x\right|}={\left|x\right|}^{0.5}$ as the default in the Mplus software package [14]. However, the loss function $\mathsf{\rho }\left(x\right)={\left|x\right|}^{0.25}$ is also available in Mplus [14]. The more general $L_p$ loss function $\mathsf{\rho }\left(x\right)={\left|x\right|}^p$ for $p>0$ was studied for IA in [12,45]. It has been shown that values of the power p smaller than 0.5 can be advantageous in some situations [12]. An interesting case is the limiting case in which p tends to zero. Effectively, $p=0$ counts the number of parameter deviations that differ from zero [47,48], resulting in the $L_0$ loss function. In the practical minimization of H involved in IA, the nondifferentiable $L_p$ loss function $\mathsf{\rho }\left(x\right)={\left|x\right|}^p$ (for $0<p\le 1$) is replaced by a differentiable approximation ${\mathsf{\rho }}_{\mathcal{D}}$ (see [5,45])

$${\mathsf{\rho }}_{\mathcal{D}}\left(x\right)={(x^2+\mathsf{\epsilon })}^{p/2},$$

(11)

where $\mathsf{\epsilon }>0$ is a tuning parameter that controls the approximation error of ${\mathsf{\rho }}_D$ for $\mathsf{\rho }$. The approximation error becomes smaller with $\mathsf{\epsilon }$ values close to zero. However, the minimization of H in IA gets more difficult when choosing too-small values of $\mathsf{\epsilon }$. Practical experience led to proposals $\mathsf{\epsilon }=0.01$ [5] or $\mathsf{\epsilon }=0.001$ [45]. The choice $\mathsf{\epsilon }=0.01$ is the default in Mplus (see [12]). It is also tempting to consider the $L_0$ loss function $\mathsf{\rho }\left(x\right)={\left|x\right|}^0$ that takes values of 1 if $x\ne 0$ and 0 for $x=0$. However, the differentiable approximation ${\mathsf{\rho }}_{\mathcal{D}}$ in (11) performs poorly for p values close to 0 because the minimization in H becomes very difficult. O’Neill and Burke [49] proposed, in a recent work related to regularized estimation, the following differentiable approximation ${\mathsf{\rho }}_{\mathcal{D}}$ of the $L_0$ loss function:

$${\mathsf{\rho }}_{\mathcal{D}}\left(x\right)=\frac{x^2}{x^2+\mathsf{\epsilon }},$$

(12)

where $\mathsf{\epsilon }>0$ is again a tuning parameter that controls the approximation error and estimation stability of the differentiable approximation. This approximation (12) of the $L_0$ loss function has not yet been investigated in IA. As IA is particularly suited to the sparse deviations in model parameters, the $L_0$ loss function should theoretically fit typical data-generating models frequently utilized in simulation studies that investigate the performance of IA.

In our experience, in the case of small $\mathsf{\epsilon }$ values, the optimization of the alignment function is very sensitive to starting values. Asparouhov and Muthén [5] remark that the linking function in invariance alignment is prone to multiple local minima. Moreover, they mention that these local minima often yield values of the linking function that are only slightly different from values at the global minimum. In the estimation of the IA approach, it is advised to choose a sequence of decreasing values of $\mathsf{\epsilon }$ in the optimization, each using the previous solution as initial values (see [50] for a similar approach). This choice guarantees better suitable starting values and a more stable estimation of IA.

3. Standard Errors in Invariance Alignment

In this section, the estimation of standard errors for the IA approach is described. IA minimizes the linking function $H(\mathbf{\mu },\mathbf{\sigma })$ with respect to $\mathbf{\mu }$ and $\mathbf{\sigma }$. Let $\mathit{\theta }=(\mathsf{\mu },\mathit{\theta })$ be the parameter vector of interest. The estimated item loadings and item intercepts across all groups are collected in a vector $\widehat{\mathit{\xi }}=({\widehat{\mathit{\xi }}}_1,\dots ,{\widehat{\mathit{\xi }}}_G)$, where ${\widehat{\mathit{\xi }}}_g$ contains the group-specific model parameter estimates from a unidimensional factor model in group $g=1,\dots ,G$. As a maximum likelihood estimate, ${\widehat{\mathit{\xi }}}_g$ is approximately multivariate normally distributed. Because of the independence of subjects across groups, a multivariate normal distribution of the input parameter $\widehat{\mathit{\xi }}$ in IA is obtained as

$$\widehat{\mathit{\xi }}-{\mathit{\xi }}_0\sim \mathsf{MVN}(\mathbf{0},{\mathit{V}}_{\mathit{\xi }}),$$

(13)

where ${\mathit{V}}_{\mathit{\xi }}$ is a block-diagonal covariance matrix, and ${\mathit{\xi }}_0$ is a population parameter.

The distribution of the IA estimates $\widehat{\mathit{\theta }}$ is now derived using the delta method in M-estimation theory [51] by relying on the implicit function theorem [5]. We assume differentiability of the optimization function because the nondifferentiable loss function $\mathsf{\rho }$ in IA is replaced by a differentiable approximation ${\mathsf{\rho }}_{\mathcal{D}}$. The IA approach minimizes $H(\mathit{\theta },\widehat{\mathit{\xi }})$, where we now highlight the dependency of the input parameters $\widehat{\mathit{\xi }}$. A parameter estimate $\widehat{\mathit{\theta }}$ is obtained by taking the partial derivative of H with respect to $\mathit{\theta }$ (i.e., $H_{\mathit{\theta }}$) and solving the nonlinear equation such that

$$H_{\mathit{\theta }}(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})=\mathbf{0}.$$

(14)

Note that there exists a population parameter ${\mathit{\theta }}_0$ such that

$$H_{\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)=\mathbf{0}.$$

(15)

Now, a Taylor expansion of $H_{\mathit{\theta }}$ around $({\mathit{\theta }}_0,{\mathit{\xi }}_0)$ can be carried out. Denote with $H_{\mathit{\theta }\mathit{\theta }}$ and $H_{\mathit{\theta }\mathit{\xi }}$ the matrices of second-order partial derivatives of $H_{\mathit{\theta }}$ with respect to $\mathit{\theta }$ and $\mathit{\xi }$, respectively. The Taylor expansion can be written as

$$H_{\mathit{\theta }}(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})=H_{\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)+H_{\mathit{\theta }\mathit{\theta }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)+H_{\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\xi }}-{\mathit{\xi }}_0)=\mathbf{0}.$$

(16)

By solving (16) for $\widehat{\mathit{\theta }}$, we have the approximation

$$\widehat{\mathit{\theta }}-{\mathit{\theta }}_0=-H_{\mathit{\theta }\mathit{\theta }}{({\mathit{\theta }}_0,{\mathit{\xi }}_0)}^{-1}{H}_{\mathit{\theta }\mathit{\xi }}({\mathit{\theta }}_0,{\mathit{\xi }}_0)(\widehat{\mathit{\xi }}-{\mathit{\xi }}_0).$$

(17)

By defining $\widehat{\mathit{A}}=-H_{\mathit{\theta }\mathit{\theta }}{(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})}^{-1}{H}_{\mathit{\theta }\mathit{\xi }}(\widehat{\mathit{\theta }},\widehat{\mathit{\xi }})$ when substituting ${\mathit{\theta }}_0$ and ${\mathit{\xi }}_0$ with $\widehat{\mathit{\theta }}$ and $\widehat{\mathit{\xi }}$, respectively, we have, by using the multivariate delta method [51],

$$\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)=\widehat{\mathit{A}}{\mathit{V}}_{\mathit{\xi }}{\widehat{\mathit{A}}}^{\top }.$$

(18)

Standard errors for elements in $\widehat{\mathit{\theta }}$ can be obtained by taking the square root of diagonal elements of $\mathsf{Var}\left(\widehat{\mathit{\theta }}\right)$ computed from (18).

4. Purpose

In this article, several implementation aspects for IA should be examined. First, it should be investigated whether the choice of the power p impacts the performance of the IA estimates. In particular, it is interesting whether there are better alternatives to the default choice $p=0.5$. Second, the deliberate choice of the tuning parameter $\mathsf{\epsilon }$ is studied. It is interesting to researchers and developers of statistical software which $\mathsf{\epsilon }$ should be selected as a default in order to produce the most reliable parameter estimates. It is expected that a larger value of $\mathsf{\epsilon }$ results in a less precise differentiable approximation and more bias for the IA estimates compared with that expected with smaller values of $\mathsf{\epsilon }$. Third, the $L_p$ loss function for $0<p\le 1$ should be compared with the newly proposed differentiable approximation of the $L_0$ loss function by O’Neill and Burke. It would be interesting to see whether the $L_0$ loss function could also be beneficial in invariance alignment. Fourth, the two different choices of the estimation of standard deviations $\mathbf{\sigma }$ (methods “NOL” and “LOG”; see Section 2) are compared. The NOL method uses deviations of ${\widehat{\mathsf{\lambda }}}_{ig,0}/{\mathsf{\sigma }}_g$ and ${\widehat{\mathsf{\lambda }}}_{ih,0}/{\mathsf{\sigma }}_h$ with respect to the loss function $\mathsf{\rho }$, while the LOG method utilizes deviations $log{\widehat{\mathsf{\lambda }}}_{ig,0}-log{\mathsf{\sigma }}_g$ and $log{\widehat{\mathsf{\lambda }}}_{ih,0}-log{\mathsf{\sigma }}_h$. Fifth, and finally, the quality of standard errors in terms of coverage is studied for the different IA estimation approaches. These research questions were answered by means of three simulation studies that are described in the next three sections.

5. Simulation Study 1: Bias and RMSE in a Three-Group Example

In Simulation Study 1, IA is studied in a case with noninvariant item intercepts and in a case with noninvariant item loadings and noninvariant item intercepts. This study focuses on bias and root mean square error (RMSE).

5.1. Method

The data-generating models (DGMs) in the simulation study mimicked the DGM used in [5]. The data were simulated from a one-dimensional factor model involving five items (i.e., $I=5$) and three groups (i.e., $G=3$). The factor variable was normally distributed with group means 0, 0.3, and 0.8, and the group variances were 1, 1.5, and 1.2, respectively. All measurement error variances were set to one in all groups and were uncorrelated with each other. The factor variable and residual variables were normally distributed.

Two DGMs were simulated that refer to a violation of measurement invariance. Group-specific item parameters that are noninvariant are referred to as DIF effects [2].

In the first DGM (i.e., DGM1), only item intercepts were noninvariant. All item loadings were set to one, and only a subset of group-specific item intercepts were simulated differently from zero. Hence, data were simulated assuming partial invariance. In the first group, the fourth item intercept was $0.5$. In the second group, the first item was $-0.5$, while the second item had an intercept of $-0.5$ in the third group.

In the second DGM (i.e., DGM2), item intercepts and item loadings were invariant. The same item intercepts as in DGM1 were used. Three group-specific item loadings were different from one. The item loading of the third item in the first group and the item loadings of the fifth item in the third group were 2.013. The second item in the second group had an item loading of 0.497.

The sample size per group was chosen as $N=250$, $N=500$, $N=1000$, or $N=2000$. IA was estimated using the NOL method (based on the $H_1$ function defined in (8)) and the LOG method (based on the $H_1^{*}$ function defined in (10)). The $L_p$ loss function was employed using the powers $p=0.5$, $p=0.25$, and $p=0.1$ and the differentiable approximation defined in (11). The $L_0$ loss function employed differentiable approximation (12). We did not consider power values $p=2$ or $p=1$ because they have been shown to result in severely biased estimates in the situation of partial invariance [12,45]. The reason is that noninvariant item intercepts (i.e., model errors) indicate a misspecified model. This kind of misspecification biases factor means if the $L_p$ loss function is with $p=2$ because all item intercepts contribute to the estimation of factor means. The situation is known from robust statistics where the unweighted mean is not robust to outlying observations. Moreover, the $L_p$ loss function with $p=1$ does not fully remove bias because it treats model errors (i.e., outlying observations) symmetrically. In contrast, the $L_p$ loss function with $p<1$ is more robust for asymmetrically distributed model errors.

All estimation methods were applied with the tuning parameters $\mathsf{\epsilon }=0.1$, $\mathsf{\epsilon }=0.01$, $\mathsf{\epsilon }=0.001$, and $\mathsf{\epsilon }=0.0001$. The choice $\mathsf{\epsilon }=0.1$ led to substantially biased parameter estimates, while the IA estimates based on $\mathsf{\epsilon }=0.0001$ had large variances. Hence, we only report findings for the tuning parameters $\mathsf{\epsilon }=0.01$ and $\mathsf{\epsilon }=0.001$.

In total, $R=1000$ replications were conducted for each cell of the simulation study. Bias, standard deviation (SD), RMSE, and relative RMSE were computed to assess the performance of the different estimators. Let ${\widehat{\theta }}_{j,r}$ be a model parameter estimate in replication $r=1,\dots ,R$ for the parameter ${\theta }_j$. The bias of the estimator ${\widehat{\theta }}_j$ was estimated with

$$\mathrm{Bias}\left({\widehat{\theta }}_j\right)=\frac{1}{R}\sum _{r=1}^R({\widehat{\theta }}_{j,r}-{\theta }_j),$$

(19)

where ${\theta }_j$ denotes the true parameter value. The SD of an estimator ${\widehat{\theta }}_j$ was calculated as

$$\mathrm{SD}\left({\widehat{\theta }}_j\right)=\sqrt{\frac{1}{R}\sum _{r=1}^R{({\widehat{\theta }}_{j,r}-{\overline{\theta }}_{j,•})}^2},\mathrm{where}{\overline{\theta }}_{j,•}=\frac{1}{R}\sum _{r=1}^R{\theta }_{j,r}.$$

(20)

The RMSE of an estimator ${\widehat{\theta }}_j$ was estimated with

$$\mathrm{RMSE}\left({\widehat{\theta }}_j\right)=\sqrt{\frac{1}{R}\sum _{r=1}^R{({\widehat{\theta }}_{j,r}-{\theta }_j)}^2}.$$

(21)

A relative RMSE can be defined by dividing the RMSE of an estimator by the RMSE of a chosen reference model. In Simulation Study 1 (and in Simulation Study 3), the Mplus default with the NOL method, $p=0.5$, and $\mathsf{\epsilon }=0.01$ is used as the reference model. To more easily grasp differences in the relative RMSE, the values were multiplied by 100. This quantity can then easily be converted into a percentage gain of a particular estimator compared with a reference model.

The entire simulation study was carried out in the R [52] software. IA was performed with the sirt::invariance.alignment() (see also [12,53,54]) function in the R package sirt (Version 4.0-19; [55]). Information about model specification can be found in the material located at https://osf.io/7kwqh/ (accessed on 17 September 2023).

5.2. Results

Table 1. Simulation Study 1: bias, standard deviation (SD), and relative root mean square error (RMSE) of the factor mean ${\mathsf{\mu }}_2$ and the factor standard deviation ${\mathsf{\sigma }}_2$ as a function of sample size N for different estimation methods in the case of noninvariant item intercepts (DGM1).

		Bias				SD				RMSE
		$\mathit{N}$				$\mathit{N}$				$\mathit{N}$
Meth	$\mathit{p}$  , $\mathsf{\epsilon }$	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000
		Factor Mean ${\mathsf{\mu }}_2$
NOL	0.5  , 0.01 ${}^{†}$	−0.067	−0.051	−0.041	−0.037	0.118	0.082	0.058	0.040	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	−0.043	−0.027	−0.016	−0.012	0.124	0.086	0.059	0.041	96.8	93.2	86.5	77.5
	0.25, 0.01	−0.050	−0.036	−0.027	−0.024	0.120	0.083	0.058	0.040	95.9	92.9	90.3	86.0
	0.25, 0.001	−0.031	−0.017	−0.009	−0.006	0.127	0.087	0.059	0.041	96.6	91.7	85.1	75.6
	0.1  , 0.01	−0.042	−0.029	−0.021	−0.019	0.121	0.083	0.058	0.040	94.5	90.4	87.1	81.3
	0.1  , 0.001	−0.026	−0.013	−0.007	−0.004	0.128	0.087	0.060	0.041	96.6	90.4	84.9	75.4
	0  , 0.01	−0.007	−0.001	0.000	0.000	0.128	0.084	0.058	0.040	94.8	86.8	82.8	73.6
	0  , 0.001	−0.006	0.000	0.001	0.000	0.133	0.089	0.063	0.043	98.4	92.3	89.6	79.1
LOG	0.5  , 0.01	−0.068	−0.052	−0.041	−0.037	0.117	0.082	0.057	0.040	100.1	100.2	100.2	100.2
	0.5  , 0.001	−0.045	−0.028	−0.016	−0.012	0.123	0.086	0.059	0.041	96.7	93.2	86.5	77.6
	0.25, 0.01	−0.051	−0.036	−0.028	−0.024	0.119	0.082	0.057	0.040	95.9	93.0	90.4	86.1
	0.25, 0.001	−0.033	−0.018	−0.009	−0.007	0.126	0.087	0.059	0.041	96.3	91.6	85.0	75.6
	0.1  , 0.01	−0.044	−0.030	−0.022	−0.019	0.120	0.082	0.058	0.040	94.5	90.5	87.2	81.4
	0.1  , 0.001	−0.028	−0.014	−0.007	−0.005	0.127	0.086	0.059	0.041	96.3	90.2	84.8	75.4
	0  , 0.01	−0.009	−0.002	0.000	0.000	0.128	0.084	0.058	0.040	94.4	86.5	82.7	73.6
	0  , 0.001	−0.008	−0.001	0.000	−0.001	0.132	0.089	0.063	0.043	97.8	92.0	89.5	79.0
		Factor SD ${\mathsf{\sigma }}_2$
NOL	0.5  , 0.01 ${}^{†}$	0.011	0.006	0.001	0.001	0.096	0.067	0.045	0.033	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.013	0.006	0.001	0.001	0.101	0.070	0.047	0.033	105.5	104.1	103.2	102.2
	0.25, 0.01	0.011	0.006	0.001	0.001	0.097	0.067	0.046	0.033	100.6	100.3	100.2	100.1
	0.25, 0.001	0.014	0.006	0.001	0.000	0.104	0.071	0.048	0.034	108.2	106.2	104.8	103.0
	0.1  , 0.01	0.012	0.006	0.001	0.001	0.097	0.067	0.046	0.033	101.1	100.5	100.3	100.1
	0.1  , 0.001	0.013	0.006	0.002	0.000	0.105	0.072	0.048	0.034	109.2	107.2	105.9	103.6
	0  , 0.01	0.013	0.006	0.001	0.001	0.105	0.070	0.047	0.033	109.1	104.7	102.6	101.1
	0  , 0.001	0.015	0.006	0.002	0.000	0.114	0.077	0.054	0.037	119.1	115.3	118.5	112.3
LOG	0.5  , 0.01	0.004	0.003	−0.001	0.000	0.096	0.067	0.046	0.033	99.1	99.5	100.1	100.0
	0.5  , 0.001	0.004	0.002	−0.001	0.000	0.101	0.069	0.047	0.033	104.2	102.9	103.1	102.2
	0.25, 0.01	0.004	0.003	−0.001	0.000	0.096	0.067	0.046	0.033	99.6	99.8	100.2	100.1
	0.25, 0.001	0.004	0.002	−0.001	0.000	0.103	0.070	0.048	0.034	106.8	104.6	104.6	103.1
	0.1  , 0.01	0.004	0.003	−0.001	0.000	0.097	0.067	0.046	0.033	100.1	100.0	100.3	100.1
	0.1  , 0.001	0.004	0.002	0.000	0.000	0.104	0.071	0.048	0.034	107.9	105.9	105.6	103.7
	0  , 0.01	0.004	0.002	−0.001	0.000	0.105	0.070	0.047	0.033	108.7	104.0	102.6	101.1
	0  , 0.001	0.004	0.001	0.000	−0.001	0.113	0.077	0.053	0.037	117.2	114.4	117.1	112.5

Note. Meth = estimation method; NOL = no logarithmized linking function for item loadings using $H_1$ in (8); LOG = logarithmized linking function for item loadings using $H_1^{*}$ in (10); p = power used in the $L_p$ loss function $\mathsf{\rho }$; $\mathsf{\epsilon }$ = tuning parameter used in the differentiable approximation of the $L_p$ loss function $\mathsf{\rho }$; absolute biases larger than 0.03 are shown with a gray background. ${}^{†}$ The Mplus defaults $p=0.5$, $\mathsf{\epsilon }=0.01$, and NOL are the reference methods in the computation of the relative RMSE. Relative RMSE values smaller than 95.0 are printed in bold font.

reports the bias, SD, and relative RMSE of the factor mean ${\mathsf{\mu }}_2$ and the factor SD ${\mathsf{\sigma }}_2$ of the second group in the DGM of noninvariant item intercepts (i.e., DGM1). It can be seen that the $L_p$ loss function with $p=0.5$, $p=0.25$, and $p=0.1$ showed some bias for ${\mathsf{\mu }}_2$. Importantly, the bias was more substantial when using $\mathsf{\epsilon }=0.01$ instead of $\mathsf{\epsilon }=0.001$. Furthermore, the extent of the bias in the estimated factor mean ${\mathsf{\mu }}_2$ decreased with increasing sample size. Notably, the pattern of the bias was similar for the NOL and LOG methods. Interestingly, the $L_0$ loss function (i.e., $p=0$) outperformed the other specifications regarding bias. While $\mathsf{\epsilon }=0.001$ would be preferable for $p=0.5$, $p=0.25$, and $p=0.1$, for $p=0$, the tuning parameter choice $\mathsf{\epsilon }=0.01$ would be preferred due to a smaller SD of the estimate. Notably, $p=0$ had a slightly increased SD compared with $p=0.5$ for the sample size $N=250$. However, this effect decreased with larger sample sizes. Moreover, it can be seen that the Mplus default $p=0.5$ and $\mathsf{\epsilon }=0.01$ could be improved in terms of relative RMSE by using $p=0.5$ and $\mathsf{\epsilon }=0.001$ for DGM1. Notably, the relative performance gains are more important in larger sample sizes. Additional smaller gains can be obtained by switching to $p=0$ and $\mathsf{\epsilon }=0.01$.

The factor SD ${\mathsf{\sigma }}_2$ was almost unbiasedly estimated in the condition of only noninvariant item intercepts. In this situation, the choice $\mathsf{\epsilon }=0.01$ can be defended over $\mathsf{\epsilon }=0.001$ for the Mplus default. The $L_0$ loss function could not outperform p values different from zero regarding the SD.

Table 2. Simulation Study 1: bias, standard deviation (SD), and relative root mean square error (RMSE) of the factor mean ${\mathsf{\mu }}_2$ and the factor standard deviation ${\mathsf{\sigma }}_2$ as a function of sample size N for different estimation methods in the case of noninvariant item intercepts and noninvariant item loadings (DGM2).

		Bias				SD				RMSE
		$\mathit{N}$				$\mathit{N}$				$\mathit{N}$
Meth	$\mathit{p}$  , $\mathsf{\epsilon }$	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000
		Factor Mean ${\mathsf{\mu }}_2$
NOL	0.5  , 0.01 ${}^{†}$	−0.016	−0.012	−0.008	−0.006	0.127	0.085	0.059	0.041	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	−0.008	−0.008	−0.004	−0.002	0.130	0.086	0.057	0.039	101.7	99.9	96.2	96.2
	0.25, 0.01	−0.007	−0.006	−0.004	−0.003	0.128	0.084	0.058	0.040	99.8	97.7	97.2	96.9
	0.25, 0.001	0.000	−0.003	−0.002	−0.001	0.132	0.086	0.058	0.039	102.5	100.3	96.7	95.8
	0.1  , 0.01	−0.003	−0.004	−0.003	−0.002	0.129	0.084	0.057	0.039	100.2	97.0	96.2	95.8
	0.1  , 0.001	0.003	−0.002	−0.001	0.000	0.133	0.086	0.058	0.039	103.8	99.9	96.8	95.7
	0  , 0.01	0.015	0.003	0.001	0.001	0.136	0.084	0.056	0.038	106.5	97.2	94.5	92.7
	0  , 0.001	0.016	0.004	0.001	0.001	0.140	0.088	0.060	0.042	109.5	102.2	100.3	102.0
LOG	0.5  , 0.01	−0.031	−0.023	−0.017	−0.014	0.120	0.082	0.057	0.039	96.7	98.5	99.7	101.6
	0.5  , 0.001	−0.021	−0.014	−0.007	−0.004	0.123	0.084	0.057	0.039	97.4	98.1	95.6	95.8
	0.25, 0.01	−0.022	−0.015	−0.011	−0.008	0.121	0.081	0.056	0.039	95.5	95.3	96.1	96.7
	0.25, 0.001	−0.014	−0.008	−0.004	−0.002	0.123	0.084	0.057	0.039	96.8	97.5	96.2	95.4
	0.1  , 0.01	−0.017	−0.012	−0.008	−0.006	0.121	0.080	0.056	0.039	95.3	94.3	95.0	95.2
	0.1  , 0.001	−0.011	−0.007	−0.003	−0.001	0.124	0.083	0.057	0.039	97.0	97.1	96.1	95.4
	0  , 0.01	−0.001	−0.001	0.000	0.001	0.126	0.080	0.056	0.038	98.0	93.1	93.1	92.5
	0  , 0.001	0.000	0.000	0.000	0.001	0.129	0.085	0.059	0.042	100.4	98.0	99.1	102.1
		Factor SD ${\mathsf{\sigma }}_2$
NOL	0.5  , 0.01 ${}^{†}$	0.183	0.135	0.108	0.093	0.134	0.094	0.061	0.041	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.142	0.072	0.042	0.028	0.161	0.098	0.059	0.038	94.6	73.7	58.2	46.0
	0.25, 0.01	0.163	0.105	0.077	0.064	0.146	0.097	0.060	0.040	96.7	86.8	78.7	73.7
	0.25, 0.001	0.132	0.056	0.028	0.017	0.175	0.104	0.058	0.037	96.5	72.2	51.8	40.4
	0.1  , 0.01	0.153	0.091	0.064	0.051	0.152	0.098	0.059	0.039	95.5	81.3	70.1	63.3
	0.1  , 0.001	0.127	0.050	0.023	0.013	0.179	0.106	0.058	0.037	96.9	71.2	50.5	39.0
	0  , 0.01	0.114	0.031	0.010	0.005	0.207	0.114	0.058	0.035	104.4	71.8	47.7	35.1
	0  , 0.001	0.113	0.028	0.007	0.003	0.214	0.120	0.064	0.040	106.8	74.9	51.6	39.4
LOG	0.5  , 0.01	0.108	0.081	0.066	0.058	0.117	0.081	0.054	0.037	70.3	69.4	68.7	67.5
	0.5  , 0.001	0.078	0.044	0.027	0.019	0.128	0.084	0.054	0.037	66.2	57.8	49.1	40.7
	0.25, 0.01	0.094	0.064	0.049	0.041	0.121	0.081	0.054	0.037	67.6	62.7	58.5	54.4
	0.25, 0.001	0.069	0.034	0.018	0.012	0.133	0.086	0.054	0.037	66.1	56.4	45.9	38.0
	0.1  , 0.01	0.086	0.055	0.041	0.034	0.124	0.081	0.053	0.036	66.7	59.8	54.2	49.1
	0.1  , 0.001	0.065	0.029	0.014	0.009	0.135	0.087	0.054	0.037	66.3	55.9	45.1	37.2
	0  , 0.01	0.047	0.013	0.004	0.003	0.147	0.083	0.051	0.035	68.3	51.4	41.2	34.6
	0  , 0.001	0.046	0.010	0.003	0.002	0.152	0.090	0.057	0.040	70.1	54.9	45.9	39.2

Note. Meth = estimation method; NOL = no logarithmized linking function for item loadings using $H_1$ in (8); LOG = logarithmized linking function for item loadings using $H_1^{*}$ in (10); p = power used in the $L_p$ loss function $\mathsf{\rho }$; $\mathsf{\epsilon }$ = tuning parameter used in the differentiable approximation of the $L_p$ loss function $\mathsf{\rho }$; absolute biases larger than 0.03 are shown with a gray background. ${}^{†}$ The Mplus defaults $p=0.5$, $\mathsf{\epsilon }=0.01$, and NOL are the reference methods in the computation of the relative RMSE. Relative RMSE values smaller than 95.0 are printed in bold font.

displays bias, SD, and relative RMSE for the factor mean and the factor SD of the second group in the condition of noninvariant item loadings and noninvariant item intercepts. In this situation, the SD estimate was biased, particularly for the smallest sample size $N=250$. Importantly, the bias was significantly reduced when using the LOG method instead of the NOL method. Overall, the $L_0$ loss function with $\mathsf{\epsilon }=0.01$ was the preferred method regarding the bias and relative RMSE for sample sizes of at least 500. Surprisingly, no bias occurred for the estimated factor mean in DGM2. However, this seems to be a coincidence of different defining factors for the bias.

Table A1. Simulation Study 1: bias, standard deviation (SD), and relative root mean square error (RMSE) of the factor mean ${\mathsf{\mu }}_3$ and the factor standard deviation ${\mathsf{\sigma }}_3$ as a function of sample size N for different estimation methods in the case of noninvariant item intercepts and noninvariant item loadings (DGM2).

		Bias				SD				RMSE
		$\mathit{N}$				$\mathit{N}$				$\mathit{N}$
Meth	$\mathit{p}$  , $\mathsf{\epsilon }$	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000
		Factor Mean ${\mathsf{\mu }}_3$
NOL	0.5  , 0.01 ${}^{†}$	−0.037	−0.031	−0.027	−0.024	0.131	0.090	0.061	0.042	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	−0.019	−0.016	−0.011	−0.009	0.142	0.098	0.062	0.043	105.4	103.6	95.4	90.0
	0.25, 0.01	−0.022	−0.019	−0.018	−0.016	0.136	0.091	0.061	0.042	101.3	97.7	95.1	92.3
	0.25, 0.001	−0.006	−0.008	−0.006	−0.005	0.148	0.101	0.064	0.043	108.8	105.7	97.0	89.5
	0.1  , 0.01	−0.015	−0.014	−0.013	−0.012	0.139	0.092	0.061	0.042	102.9	97.2	93.7	89.8
	0.1  , 0.001	−0.002	−0.005	−0.004	−0.003	0.151	0.101	0.065	0.043	111.2	105.3	97.5	89.7
	0  , 0.01	0.016	0.007	0.000	0.000	0.163	0.098	0.062	0.041	120.1	103.2	93.9	85.4
	0  , 0.001	0.016	0.008	0.001	0.000	0.169	0.107	0.070	0.048	125.0	111.9	104.8	98.2
LOG	0.5  , 0.01	−0.066	−0.053	−0.044	−0.038	0.126	0.087	0.059	0.041	104.4	106.3	110.6	115.8
	0.5  , 0.001	−0.045	−0.030	−0.020	−0.014	0.136	0.094	0.061	0.043	105.5	103.4	97.0	92.1
	0.25, 0.01	−0.049	−0.038	−0.030	−0.026	0.130	0.088	0.059	0.041	102.6	100.0	100.4	100.5
	0.25, 0.001	−0.032	−0.020	−0.012	−0.008	0.141	0.096	0.063	0.043	106.6	102.8	96.9	90.0
	0.1  , 0.01	−0.041	−0.031	−0.025	−0.021	0.133	0.088	0.060	0.041	102.7	97.9	97.0	95.2
	0.1  , 0.001	−0.026	−0.015	−0.009	−0.006	0.145	0.097	0.064	0.043	108.5	102.9	97.0	89.7
	0  , 0.01	−0.006	−0.001	−0.002	−0.002	0.154	0.095	0.062	0.041	113.1	98.9	93.1	85.2
	0  , 0.001	−0.006	0.000	−0.002	−0.001	0.160	0.103	0.069	0.047	117.4	107.6	104.0	97.7
		Factor SD ${\mathsf{\sigma }}_3$
NOL	0.5  , 0.01 ${}^{†}$	0.051	0.034	0.028	0.022	0.102	0.069	0.048	0.033	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.046	0.023	0.015	0.009	0.116	0.075	0.049	0.033	109.5	100.7	92.7	85.5
	0.25, 0.01	0.048	0.029	0.022	0.016	0.107	0.071	0.048	0.033	103.0	98.5	95.3	92.3
	0.25, 0.001	0.045	0.020	0.011	0.006	0.122	0.078	0.050	0.033	114.1	103.2	92.3	83.8
	0.1  , 0.01	0.047	0.026	0.019	0.014	0.109	0.071	0.048	0.033	104.4	98.0	93.3	89.3
	0.1  , 0.001	0.044	0.018	0.010	0.005	0.124	0.079	0.050	0.033	115.1	104.2	92.8	83.5
	0  , 0.01	0.040	0.014	0.006	0.003	0.132	0.079	0.048	0.032	121.4	103.2	87.3	80.1
	0  , 0.001	0.040	0.015	0.007	0.003	0.138	0.086	0.054	0.035	126.4	112.7	98.4	89.2
LOG	0.5  , 0.01	0.007	0.002	0.003	0.001	0.097	0.066	0.046	0.032	85.5	85.4	83.8	80.7
	0.5  , 0.001	0.008	0.003	0.003	0.002	0.109	0.071	0.048	0.032	95.7	92.1	86.9	81.6
	0.25, 0.01	0.007	0.003	0.003	0.001	0.102	0.067	0.047	0.032	89.8	86.9	84.6	80.8
	0.25, 0.001	0.008	0.003	0.003	0.002	0.114	0.073	0.049	0.033	100.4	94.8	88.1	81.8
	0.1  , 0.01	0.008	0.003	0.003	0.001	0.105	0.068	0.047	0.032	92.0	87.7	84.8	80.8
	0.1  , 0.001	0.008	0.004	0.003	0.002	0.117	0.074	0.049	0.033	102.8	96.1	88.9	82.0
	0  , 0.01	0.010	0.003	0.003	0.001	0.124	0.074	0.047	0.032	108.9	94.9	85.6	79.5
	0  , 0.001	0.011	0.004	0.003	0.002	0.128	0.081	0.053	0.036	112.9	104.1	96.6	89.3

Note. Meth = estimation method; NOL = no logarithmized linking function for item loadings using $H_1$ in (8); LOG = logarithmized linking function for item loadings using $H_1^{*}$ in (10); p = power used in the $L_p$ loss function $\mathsf{\rho }$; $\mathsf{\epsilon }$ = tuning parameter used in the differentiable approximation of the $L_p$ loss function $\mathsf{\rho }$; absolute biases larger than 0.03 are shown with a gray background. ${}^{†}$ The Mplus defaults $p=0.5$, $\mathsf{\epsilon }=0.01$, and NOL are the reference methods in the computation of the relative RMSE. Relative RMSE values smaller than 95.0 are printed in bold font.

in Appendix A reveals that the factor mean of the third group in DGM2 also provided biased estimates. Again, in this case, $p=0$ resolved the issue, in particular for larger sample sizes.

As a preliminary conclusion from Simulation Study 1, one could argue that the case of noninvariant item loadings can induce bias in parameter estimates in default implementations of IA. The bias can be reduced by using the LOG method instead of the default NOL method. There is a tendency that the choice $\mathsf{\epsilon }=0.001$ outperformed $\mathsf{\epsilon }=0.01$ for $p=0.5$. Finally, the $L_0$ loss function (i.e., $p=0$) had satisfactory performance for $\mathsf{\epsilon }=0.01$. However, it came at the price of increased variance in smaller sample sizes.

6. Simulation Study 2: Coverage Rates in a Three-Group Example

The second example, Simulation Study 2, investigated the assessment of coverage rates for the different IA estimation methods.

6.1. Method

The same DGMs as in Simulation Study 1 were employed to simulate the data (see Section 5). The standard error computation described in Section 3 was applied for all estimators used in Simulation Study 1. Confidence intervals at a confidence level of 95% were computed using a normal distribution approximation (i.e., the estimated confidence interval was $\widehat{\theta }\pm 1.96\times \mathrm{SE}\left(\widehat{\theta }\right)$). The coverage rate at a confidence level of 95% was computed as the percentage of the events that an estimated confidence interval covers for the true parameter value. Coverage rates were considered acceptable if they were not smaller than 91.0 or larger than 98.0 (see [56]).

In total, 5000 replications were conducted in each cell of the simulation. The IA method with standard error estimates of model parameters was again estimated with the sirt::invariance.alignment() function that is contained in the R [52] package sirt (Version 4.0-19; [55]). The R code used for this simulation can be found at https://osf.io/7kwqh/ (accessed on 17 September 2023).

6.2. Results

Table 3. Simulation Study 2: coverage rates for factor means ${\mathsf{\mu }}_2$ and ${\mathsf{\mu }}_3$ and factor standard deviations ${\mathsf{\sigma }}_2$ and ${\mathsf{\sigma }}_3$ as a function of sample size N for different estimation methods.

		${\mathsf{\mu }}_2$				${\mathsf{\mu }}_3$				${\mathsf{\sigma }}_2$				${\mathsf{\sigma }}_3$
		$\mathit{N}$				$\mathit{N}$				$\mathit{N}$				$\mathit{N}$
Meth	$\mathit{p}$  , $\mathsf{\epsilon }$	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000
		Noninvariant Item Intercepts (DGM1)
NOL	0.5  , 0.01	92.7	91.4	88.0	84.4	92.5	90.6	89.2	85.7	95.6	95.5	95.5	95.2	95.8	95.4	95.4	94.9
	0.5  , 0.001	95.8	95.6	94.8	94.9	95.9	95.9	95.8	95.0	97.5	97.4	96.8	96.1	98.1	97.3	96.9	96.0
	0.25, 0.01	94.5	93.6	91.6	90.0	94.4	93.1	92.5	90.9	95.8	95.6	95.6	95.2	96.1	95.7	95.5	95.0
	0.25, 0.001	96.2	96.4	95.8	95.7	96.5	96.8	96.5	96.0	97.8	97.4	97.1	96.5	98.2	97.6	97.3	96.3
	0.1  , 0.01	94.9	94.2	92.7	92.2	95.2	94.0	93.5	92.5	96.1	95.7	95.6	95.3	96.4	95.8	95.6	95.0
	0.1  , 0.001	96.2	96.5	96.0	96.0	96.6	97.1	96.7	96.2	97.9	97.5	97.3	96.6	98.0	97.6	97.5	96.5
	0  , 0.01	96.3	96.2	95.3	95.6	96.5	96.6	95.9	95.5	97.2	96.9	96.4	95.6	97.4	96.7	96.5	95.5
	0  , 0.001	96.2	96.7	96.5	96.6	96.8	97.4	97.3	97.3	97.6	97.5	97.6	97.6	97.9	97.7	97.4	97.1
LOG	0.5  , 0.01	92.5	91.3	87.8	84.3	92.0	90.3	88.9	85.4	95.6	95.3	95.5	95.1	95.6	95.3	95.4	95.0
	0.5  , 0.001	95.7	95.5	94.8	94.8	95.7	95.7	95.7	94.9	97.5	97.1	96.8	96.1	97.8	97.2	96.9	96.0
	0.25, 0.01	94.3	93.5	91.5	90.0	93.9	92.9	92.2	90.7	95.7	95.3	95.6	95.2	95.9	95.4	95.5	95.0
	0.25, 0.001	95.9	96.3	95.8	95.7	96.2	96.6	96.5	95.9	97.6	97.4	97.1	96.4	97.8	97.5	97.2	96.3
	0.1  , 0.01	94.7	94.1	92.6	92.2	94.7	93.8	93.3	92.4	95.9	95.4	95.6	95.2	96.0	95.5	95.5	95.0
	0.1  , 0.001	96.0	96.5	96.0	96.0	96.4	97.1	96.7	96.1	97.6	97.4	97.2	96.6	97.9	97.5	97.3	96.4
	0  , 0.01	96.3	96.2	95.3	95.6	96.4	96.6	95.8	95.5	97.0	96.7	96.3	95.5	97.0	96.7	96.5	95.5
	0  , 0.001	96.2	96.7	96.4	96.7	96.7	97.3	97.3	97.3	97.7	97.4	97.7	97.4	97.5	97.5	97.4	97.0
		Noninvariant Item Intercepts and Noninvariant Item Loadings (DGM2)
NOL	0.5  , 0.01	96.0	95.6	95.1	94.8	94.5	94.1	93.2	91.7	84.9	78.6	64.4	41.7	96.2	94.6	93.4	91.6
	0.5  , 0.001	96.9	96.7	96.0	96.1	96.6	96.6	96.2	96.1	92.2	95.4	95.2	93.0	97.3	96.9	96.6	96.2
	0.25, 0.01	96.4	95.9	95.4	95.2	95.6	95.1	94.3	93.8	89.0	89.1	81.8	69.5	96.6	95.4	94.6	93.7
	0.25, 0.001	96.7	96.7	96.4	96.3	96.7	96.9	96.9	96.5	91.0	95.4	96.0	95.3	97.2	96.8	96.9	96.4
	0.1  , 0.01	96.4	96.0	95.3	95.3	95.9	95.4	94.9	94.4	89.7	92.4	86.9	79.1	96.6	95.6	94.9	94.2
	0.1  , 0.001	96.8	96.5	96.4	96.4	96.7	96.9	97.0	96.7	90.6	95.5	96.5	96.0	96.9	96.8	96.8	96.6
	0  , 0.01	95.9	96.4	95.6	95.6	95.7	96.8	96.2	95.6	87.5	94.7	96.0	95.3	95.3	96.3	96.1	95.8
	0  , 0.001	95.9	96.4	96.6	96.7	95.7	96.9	97.2	97.4	86.5	94.6	96.9	96.6	95.5	96.3	97.0	96.9
LOG	0.5  , 0.01	95.3	94.9	93.8	93.4	91.8	91.0	89.2	85.6	92.7	88.7	81.6	70.1	96.0	95.5	95.2	95.2
	0.5  , 0.001	96.6	96.5	95.9	95.8	95.5	96.2	95.9	95.7	96.5	96.9	95.9	94.8	97.5	97.0	96.7	96.2
	0.25, 0.01	95.8	95.4	94.6	94.3	93.7	93.3	92.5	90.8	94.8	93.3	88.9	83.4	96.2	95.8	95.3	95.2
	0.25, 0.001	96.6	96.6	96.2	96.2	95.9	96.7	96.5	96.5	96.4	97.0	96.7	96.0	97.2	97.0	96.9	96.5
	0.1  , 0.01	96.0	95.8	94.9	94.7	94.5	94.3	93.4	92.7	95.5	94.8	91.4	87.7	96.3	96.0	95.4	95.2
	0.1  , 0.001	96.7	96.6	96.3	96.4	95.9	96.7	96.8	96.7	96.3	96.9	97.0	96.3	96.9	97.1	97.0	96.6
	0  , 0.01	96.1	96.5	95.6	95.5	95.3	96.8	96.1	95.5	95.2	96.6	96.1	95.3	95.7	96.5	96.1	95.8
	0  , 0.001	96.2	96.6	96.6	96.7	95.1	97.0	97.1	97.3	94.7	96.5	96.8	96.5	96.0	96.6	96.8	96.9

Note. Meth = estimation method; NOL = no logarithmized linking function for item loadings using $H_1$ in (8); LOG = logarithmized linking function for item loadings using $H_1^{*}$ in (10); p = power used in the $L_p$ loss function $\mathsf{\rho }$; $\mathsf{\epsilon }$ = tuning parameter used in the differentiable approximation of the $L_p$ loss function $\mathsf{\rho }$; coverage rates smaller than 91.0 or larger than 98.0 are shown with a gray background.

shows coverage rates for factor means ${\mathsf{\mu }}_2$ and ${\mathsf{\mu }}_3$ and factor SDs ${\mathsf{\sigma }}_2$ and ${\mathsf{\sigma }}_3$. Overall, the coverage rates were acceptable. Undercoverage was observed when parameter estimates were biased (e.g., $p=0.5$, $\mathsf{\epsilon }=0.01$). For approximately unbiased point estimates, standard errors based on the delta method can be reliably used. There is no need to rely on computationally more demanding standard error estimates like jackknife or bootstrap. Interestingly, the coverage rates were also satisfactory for the newly proposed loss function with the power $p=0$.

7. Simulation Study 3: Bias and RMSE in a Six-Group Example

In the last example, Simulation Study 3, different estimation methods of IA were examined for DGMs involving six groups and four items.

7.1. Method

The data were simulated from a one-dimensional factor model involving four items (i.e., $I=4$) and six groups (i.e., $G=6$). The factor variable was normally distributed with group means 0, −0.27, −0.46, 0.11, 0.21, and 0.49, and the group variances were 1, 0.95, 0.87, 1.23, 1.1, and 0.99, respectively. All measurement error variances were set to one in all groups and uncorrelated with each other. The factor variable and residual variables were normally distributed.

Two DGMs were simulated that refer to a violation of measurement invariance. Invariance only appeared in item loadings, while item intercepts had invariant parameters.

In the first DGM of Simulation Study 3 (i.e., DGM3), the DIF effects in item loadings ${\mathsf{\lambda }}_{ig}$ ($i=1,\dots ,I$, …) were unidirectional. That is, item loadings with DIF loadings were all larger than one, while the invariant item parameters had loadings equal to one. The item loadings with DIF effects were as follows: ${\mathsf{\lambda }}_{12}=2.014$ (i.e., first item in second group), ${\mathsf{\lambda }}_{21}=1.733$, ${\mathsf{\lambda }}_{23}=2.014$, and ${\mathsf{\lambda }}_{36}=2.117$. The item intercepts were all set to 0 in DGM3.

In the second DGM of Simulation Study 3 (i.e., DGM4), the DIF effects in item loadings were directional. That is, item loadings with DIF effects could be smaller or larger than one. The item loadings with DIF effects were as follows: ${\mathsf{\lambda }}_{12}=0.497$, ${\mathsf{\lambda }}_{21}=1.733$, ${\mathsf{\lambda }}_{23}=2.014$, and ${\mathsf{\lambda }}_{36}=2.117$. Like in DGM3, the item intercepts were all set to 0 in DGM4.

As in Simulation Study 1 and Simulation Study 2, the sample size per group was chosen as $N=250$, $N=500$, $N=1000$, or $N=2000$. The same IA estimation methods as in the other two studies were utilized. To summarize the performance of the parameter estimates across the six groups, average absolute bias, average SD, and the average relative RMSE were computed, where the average was calculated for factor means $\mathsf{\mu }$ and factor SDs $\mathsf{\sigma }$, separately.

Again, the simulation study was carried out in the R [52] software. IA was estimated using the sirt::invariance.alignment() function in the R package sirt (Version 4.0-19; [55]). Replication material can be found at https://osf.io/7kwqh/ (accessed on 17 September 2023).

7.2. Results

Table 4. Simulation Study 3: average absolute bias, average standard deviation (average SD), and average relative root mean square error (ARRMSE) for factor means $\mathsf{\mu }$ and factor standard deviations $\mathsf{\sigma }$ as a function of sample size N for different estimation methods for unidirectional effects in noninvariance in item loadings (DGM3).

		Average Absolute Bias				Average SD				ARRMSE
		$\mathit{N}$				$\mathit{N}$				$\mathit{N}$
Meth	$\mathit{p}$  , $\mathsf{\epsilon }$	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000
		Factor Means $\mathsf{\mu }$
NOL	0.5  , 0.01 ${}^{†}$	0.047	0.030	0.025	0.021	0.108	0.071	0.050	0.034	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.034	0.017	0.011	0.007	0.108	0.071	0.048	0.034	95.1	92.9	87.4	83.2
	0.25, 0.01	0.033	0.019	0.015	0.011	0.104	0.070	0.048	0.034	91.4	92.1	89.9	87.7
	0.25, 0.001	0.025	0.011	0.007	0.003	0.107	0.071	0.048	0.033	91.9	91.4	84.9	81.0
	0.1  , 0.01	0.028	0.015	0.012	0.008	0.102	0.069	0.048	0.033	88.9	89.9	87.1	84.6
	0.1  , 0.001	0.022	0.009	0.005	0.002	0.107	0.071	0.048	0.033	91.6	90.9	85.1	81.1
	0  , 0.01	0.012	0.004	0.003	0.001	0.105	0.069	0.047	0.033	88.4	87.1	82.2	79.1
	0  , 0.001	0.012	0.005	0.003	0.001	0.111	0.074	0.051	0.035	93.5	93.7	90.1	84.7
LOG	0.5  , 0.01	0.003	0.003	0.003	0.003	0.091	0.065	0.046	0.032	75.9	82.0	80.6	78.7
	0.5  , 0.001	0.003	0.003	0.002	0.002	0.094	0.067	0.046	0.033	78.8	84.4	80.9	79.1
	0.25, 0.01	0.003	0.003	0.002	0.003	0.092	0.065	0.046	0.032	76.5	82.1	80.8	78.7
	0.25, 0.001	0.002	0.003	0.002	0.002	0.096	0.068	0.047	0.033	80.6	85.8	81.6	79.3
	0.1  , 0.01	0.003	0.003	0.002	0.002	0.092	0.066	0.046	0.032	77.0	82.5	80.7	78.8
	0.1  , 0.001	0.002	0.003	0.002	0.001	0.098	0.069	0.047	0.033	81.6	87.3	82.4	79.9
	0  , 0.01	0.002	0.002	0.002	0.001	0.098	0.068	0.046	0.033	81.8	85.5	81.4	78.9
	0  , 0.001	0.002	0.003	0.002	0.001	0.104	0.074	0.051	0.035	86.8	92.4	89.7	84.7
		Factor SDs $\mathsf{\sigma }$
NOL	0.5  , 0.01 ${}^{†}$	0.126	0.091	0.072	0.063	0.118	0.071	0.048	0.035	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.087	0.053	0.031	0.021	0.120	0.073	0.049	0.034	86.6	78.4	67.9	55.6
	0.25, 0.01	0.084	0.060	0.043	0.035	0.111	0.069	0.048	0.034	80.9	79.3	74.6	68.8
	0.25, 0.001	0.064	0.037	0.019	0.011	0.116	0.073	0.049	0.033	77.4	71.2	61.0	49.4
	0.1  , 0.01	0.071	0.049	0.033	0.026	0.109	0.068	0.047	0.034	75.6	73.0	67.1	60.0
	0.1  , 0.001	0.056	0.031	0.015	0.008	0.117	0.073	0.049	0.033	75.4	68.8	59.3	48.1
	0  , 0.01	0.031	0.018	0.008	0.004	0.110	0.069	0.046	0.032	66.7	61.9	54.6	45.4
	0  , 0.001	0.028	0.016	0.006	0.002	0.118	0.077	0.052	0.035	70.5	68.3	60.8	49.8
LOG	0.5  , 0.01	0.015	0.010	0.009	0.007	0.090	0.062	0.044	0.032	53.3	54.7	52.1	46.4
	0.5  , 0.001	0.010	0.006	0.004	0.002	0.097	0.066	0.045	0.032	56.8	57.7	53.0	45.7
	0.25, 0.01	0.011	0.007	0.006	0.005	0.091	0.062	0.044	0.032	53.4	54.6	51.8	45.6
	0.25, 0.001	0.008	0.004	0.002	0.001	0.100	0.068	0.046	0.032	58.1	59.1	53.6	46.0
	0.1  , 0.01	0.009	0.006	0.005	0.004	0.092	0.063	0.044	0.032	53.8	54.6	51.7	45.3
	0.1  , 0.001	0.006	0.005	0.002	0.001	0.101	0.069	0.047	0.033	59.0	60.1	54.3	46.2
	0  , 0.01	0.003	0.007	0.004	0.002	0.101	0.067	0.046	0.032	58.8	58.6	53.2	45.4
	0  , 0.001	0.003	0.006	0.002	0.001	0.108	0.076	0.052	0.035	63.0	66.4	60.3	49.9

Note. Meth = estimation method; NOL = no logarithmized linking function for item loadings using $H_1$ in (8); LOG = logarithmized linking function for item loadings using $H_1^{*}$ in (10); p = power used in the $L_p$ loss function $\mathsf{\rho }$; $\mathsf{\epsilon }$ = tuning parameter used in the differentiable approximation of the $L_p$ loss function $\mathsf{\rho }$; values of average absolute bias larger than 0.03 are shown with a gray background. ${}^{†}$ The Mplus defaults $p=0.5$, $\mathsf{\epsilon }=0.01$, and NOL are the reference methods in the computation of the relative RMSE. ARRMSE values smaller than 95.0 are printed in bold font.

displays average absolute bias, average SD, and average relative RMSE for factor means $\mathsf{\mu }$ and factor SDs $\mathsf{\sigma }$ for the DGM with unidirectional effects of noninvariance (i.e., DGM3). As expected from Simulation Study 1, noninvariant item loadings mainly impacted factor standard deviations. It was obtained that factor SDs had a larger absolute bias for the (default) NOL method compared with the LOG method. However, absolute bias decreased with larger sample sizes and by choosing the tuning parameter $\mathsf{\epsilon }=0.001$ instead of $\mathsf{\epsilon }=0.01$. Furthermore, the LOG method had a much smaller bias in estimated factor SDs. This finding also translated into findings for the average absolute RMSE of the $\mathsf{\sigma }$ estimates. Interestingly, the LOG method was also preferred for the estimated factor means. The LOG estimates for $\mathsf{\mu }$ had, on average, smaller SDs than the NOL estimates.

Similar findings were obtained in the case of the bidirectional DIF effects in item loadings (DGM4) that are displayed in

Table A2. Simulation Study 3: average absolute bias, average standard deviation (average SD), and average relative root mean square error (ARRMSE) for factor means $\mathsf{\mu }$ and factor standard deviations $\mathsf{\sigma }$ as a function of sample size N for different estimation methods for bidirectional effects in noninvariance in item loadings (DGM4).

		Average Absolute Bias				Average SD				ARRMSE
		$\mathit{N}$				$\mathit{N}$				$\mathit{N}$
Meth	$\mathit{p}$  , $\mathsf{\epsilon }$	250	500	1000	2000	250	500	1000	2000	250	500	1000	2000
		Factor Means $\mathsf{\mu }$
NOL	0.5  , 0.01 ${}^{†}$	0.029	0.023	0.019	0.016	0.105	0.073	0.051	0.036	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.020	0.014	0.009	0.006	0.107	0.073	0.051	0.035	99.1	95.1	93.0	88.0
	0.25, 0.01	0.021	0.016	0.012	0.010	0.103	0.072	0.051	0.035	95.7	95.0	94.7	91.8
	0.25, 0.001	0.015	0.010	0.006	0.003	0.109	0.074	0.051	0.035	99.2	94.5	91.7	86.3
	0.1  , 0.01	0.018	0.013	0.010	0.008	0.103	0.072	0.050	0.035	94.6	93.3	92.9	89.4
	0.1  , 0.001	0.013	0.009	0.004	0.003	0.109	0.074	0.051	0.035	99.2	94.2	91.7	86.6
	0  , 0.01	0.009	0.005	0.002	0.002	0.110	0.073	0.050	0.035	98.7	92.0	89.2	85.1
	0  , 0.001	0.010	0.004	0.002	0.001	0.116	0.077	0.054	0.037	104.5	97.9	96.6	91.7
LOG	0.5  , 0.01	0.008	0.004	0.005	0.004	0.094	0.068	0.048	0.034	84.7	86.9	87.2	84.1
	0.5  , 0.001	0.006	0.002	0.002	0.002	0.099	0.070	0.049	0.034	89.2	88.2	87.9	84.2
	0.25, 0.01	0.006	0.003	0.003	0.002	0.094	0.069	0.048	0.034	84.9	87.3	87.3	84.0
	0.25, 0.001	0.005	0.002	0.001	0.001	0.102	0.071	0.049	0.035	91.7	89.7	88.6	85.0
	0.1  , 0.01	0.005	0.003	0.003	0.002	0.096	0.069	0.049	0.034	85.9	87.3	87.4	84.0
	0.1  , 0.001	0.005	0.002	0.001	0.001	0.103	0.072	0.050	0.035	92.7	90.5	89.0	85.6
	0  , 0.01	0.003	0.002	0.001	0.001	0.105	0.071	0.049	0.035	93.8	89.9	88.4	84.7
	0  , 0.001	0.004	0.001	0.001	0.001	0.110	0.076	0.053	0.037	98.5	95.6	95.7	91.8
		Factor SDs $\mathsf{\sigma }$
NOL	0.5  , 0.01 ${}^{†}$	0.083	0.062	0.050	0.042	0.099	0.067	0.048	0.033	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$	100 ${}^{†}$
	0.5  , 0.001	0.061	0.039	0.023	0.014	0.104	0.071	0.050	0.033	93.7	89.6	80.3	69.3
	0.25, 0.01	0.062	0.044	0.033	0.026	0.098	0.067	0.048	0.033	89.8	88.0	84.7	79.6
	0.25, 0.001	0.047	0.029	0.014	0.008	0.105	0.073	0.049	0.033	89.6	86.3	76.2	65.6
	0.1  , 0.01	0.054	0.037	0.026	0.020	0.098	0.067	0.047	0.033	87.3	84.3	79.6	73.5
	0.1  , 0.001	0.043	0.025	0.011	0.006	0.106	0.073	0.050	0.033	89.0	85.3	75.5	64.9
	0  , 0.01	0.030	0.015	0.007	0.003	0.105	0.069	0.047	0.032	85.2	78.7	70.0	62.3
	0  , 0.001	0.029	0.013	0.005	0.002	0.113	0.078	0.053	0.036	90.9	87.4	79.2	70.9
LOG	0.5  , 0.01	0.029	0.023	0.018	0.016	0.089	0.063	0.044	0.031	76.7	77.2	75.0	74.3
	0.5  , 0.001	0.019	0.012	0.008	0.006	0.097	0.067	0.046	0.032	78.7	76.4	70.4	64.4
	0.25, 0.01	0.021	0.015	0.012	0.010	0.091	0.063	0.045	0.031	74.8	73.5	70.2	66.4
	0.25, 0.001	0.013	0.007	0.005	0.003	0.100	0.069	0.047	0.032	80.1	76.9	70.7	63.8
	0.1  , 0.01	0.017	0.012	0.009	0.007	0.091	0.064	0.045	0.031	74.2	72.6	68.9	63.9
	0.1  , 0.001	0.011	0.005	0.003	0.003	0.102	0.069	0.048	0.032	80.9	77.5	71.1	63.9
	0  , 0.01	0.004	0.002	0.001	0.001	0.101	0.068	0.046	0.031	79.4	74.9	68.4	61.7
	0  , 0.001	0.004	0.001	0.001	0.001	0.109	0.076	0.053	0.036	85.8	84.2	78.4	70.6

Note. Meth = estimation method; NOL = no logarithmized linking function for item loadings using $H_1$ in (8); LOG = logarithmized linking function for item loadings using $H_1^{*}$ in (10); p = power used in the $L_p$ loss function $\mathsf{\rho }$; $\mathsf{\epsilon }$ = tuning parameter used in the differentiable approximation of the $L_p$ loss function $\mathsf{\rho }$; values of average absolute bias larger than 0.03 are shown with a gray background. ${}^{†}$ The Mplus defaults $p=0.5$, $\mathsf{\epsilon }=0.01$, and NOL are the reference methods in the computation of the relative RMSE. ARRMSE values smaller than 95.0 are printed in bold font.

in Appendix B. The LOG method was preferred over the NOL method regarding the estimation of factor SDs $\mathsf{\sigma }$. Furthermore, LOG resulted in slightly better estimates than the NOL method for factor means $\mathsf{\mu }$. IA with $p=0.5$ should preferably choose the tuning parameter $\mathsf{\epsilon }=0.001$ instead of $\mathsf{\epsilon }=0.01$.

8. Discussion and Conclusions

In this article, we critically discussed implementation aspects in IA. Because IA is now widely applied in the social sciences, researchers should opt for appropriate estimation methods. We derived recommendations for software implementation and practical application of IA through three simulation studies.

In IA, the loss function $\mathsf{\rho }\left(x\right)=\sqrt{\left|x\right|}={\left|x\right|}^{0.5}$ is the default choice in the popular Mplus software. A differentiable approximation of this loss function uses the tuning parameter $\mathsf{\epsilon }=0.01$ as a default in this software. Our simulations revealed that this default choice can induce bias in estimated factor means and factor standard deviations. The bias can be reduced by switching to the tuning parameter $\mathsf{\epsilon }=0.001$. Notably, biases in IA were particularly pronounced in small to moderate samples (i.e., $N=250$ persons per group). It turned out that bias in estimated factor standard deviations occurred in the presence of noninvariant item loadings. This bias can be reduced by using a modified IA optimization function in which logarithmized item loadings are aligned (i.e., the LOG method described in this paper). In general, we found that the LOG method generally improves the default NOL method in IA (which uses no logarithmized item loadings) in the situations in which bias occurred and performed comparably to NOL in all other situations. Furthermore, the $L_0$ loss function recently proposed by O’Neill and Burke (i.e., $p=0$) showed the least bias across all simulated conditions. This method can be regarded as the frontrunner across all simulation conditions when used with the tuning parameter $\mathsf{\epsilon }=0.01$. Finally, statistical inference based on the delta method performed satisfactorily for all approximately unbiased estimates in terms of coverage rates.

In future research, the generalizability of the findings of this study to more groups or more items [17] can be examined. Moreover, implementation aspects of IA could also be investigated for dichotomous or ordinal items [19,20]. In particular, the performance of IA in small samples [57] requires additional consideration. It might be that regularized estimation [58,59,60,61] or confirmatory factor analysis estimation that uses robust loss functions (i.e., model-robust estimation; see [62,63]) have advantages over IA in small samples.