Editorial to the Special Issue “Feature Papers in Psychometrics and Educational Measurement”

The Special Issue “Feature Papers in Psychometrics and Educational Measurement” (https://www.mdpi.com/journal/psych/specialissues/featurepaperspsychometrics, accessed on 6 September 2023) targeted manuscripts from the field of psychological and educational measurement. This Special Issue is part of the Section “Psychometrics and Educational Measurement” (https://www.mdpi.com/journal/psych/sections/PsychometricsEducationalMeasurement (accessed on 6 September 2023)) in the Psych journal. This Section “Psychometrics and Educational Measurement” seeks to publish original methodological and applied research that targets measurement in psychology and education.

The articles published in this Special Issue are discussed below in the chronological order of their publication date.

The article by Temel, Rietz, Machunsky, and Bedersdorfer (2022) titled “Examining and improving the gender and language DIF in the VERA 8 tests” [1] examines differential item functioning (DIF) regarding gender and language groups in a German educational assessment study. The authors study DIF using the Rasch and the two-parameter logistic (2PL) item response theory (IRT) model. The model fit has since been improved when estimating a 2PL model with a partial invariance assumption; that is, some of the item parameters were allowed to differ across groups. Different DIF treatments in statistical analyses were less consequential on average scores and ability classifications but had some impact on the proportion of students classified in the highest or lowest ability level. I tend to view DIF as a diagnostic tool to investigate the heterogeneous behavior of items across groups. However, in my view, DIF is unrelated to a potential bias in group differences in scores. If group differences differ across items due to the presence of DIF, researchers would then have to decide how to identify a “true” difference. In the Rasch model and in the case of two groups, this question boils down to finding (or defining) a location measure of the variable defined by the item difficulty differences of the two groups. In robust statistics, there are a multitude of alternatives to define a location measure, such as the mean, median, symmetrically and asymmetrically trimmed mean, Huber mean, etc. Treating some items as partially invariant corresponds to choosing a particular robust location measure [2]. I argue that statistical (or psychometric) reasoning cannot help resolve the issue of defining an appropriate location measure. Alternatively, I would always opt for deciding which items have construct-relevant or construct-irrelevant DIF. This is a substantive decision that can only be made by content experts familiar with the test instrument. Only items with construct-irrelevant DIF should be eliminated from group comparisons.

The article by Martínez-Huertas, Moreno, Olmos, Martínez-Mingo, and Jorge-Botana (2022) titled “A failed cross-validation study on the relationship between LIWC linguistic indicators and personality: Exemplifying the lack of generalizability of exploratory studies” [3] is a replication study to study the relationship between the “Big 5 personality traits” and linguistic indicators. In a sample of $N=643$ Spanish undergraduate students, it was found that language and personality relationships were not generalizable in cross-validation studies. This study proves that cross-validation analysis (see [4]) is useful in assessing the stability and replicability of model parameter estimates.

The article by Feuerstahler (2023) titled “Scale type revisited: Some misconceptions, misinterpretations, and recommendations” [5] critically reflects on Steven’s classification of scales into the nominal, ordinal, interval, and ratio types. The author concludes that the current notations of scale types in representational measurement theory are of limited use in research. Moreover, Feuerstahler argues that scale types should play a minor role in teacher introductory statistics courses in the social sciences. The author contends that it is not the scale type, but the distinction between discrete and continuous variables which is relevant for choosing an appropriate statistical method. I agree with almost all of the arguments presented in the article and believe that scale types should be withdrawn from the curriculum of introductory statistics courses because the axiomatic definition of scale types is unrelated to the choosing of a statistical method. I also fully agree with the argument of Feuerstahler that the estimation and assessment of the model fit of certain latent variable models, such as the Rasch model or other IRT models, ensure the interval scale type. The factor model (i.e., the measurement model) is based on a statistical model involving probabilistic relationships among items, whereas axioms in representational measurement theory operate on scores (i.e., a scale). It is in my opinion that using a variable (or a score from a latent variable model) in a statistical analysis refers to the question of the adequacy of certain modeling assumptions, which has no relation with scale types. It will also likely be the case that the means with which to treat a variable (i.e., using raw scores, rank scores, or any other transformation from the original scores) can differ across analyses depending on the purpose (see also [6]).

The article by Vicente (2023) titled “Evaluating the effect of planned missing designs in structural equation model fit measures” [7] addresses the assessment of the model fit of structural equation models (SEMs) for the most frequently used model fit statistics. It was found that the comparative fit index (CFI), the standardized root mean square residual (SRMR) index, and the Tucker–Lewis index (TLI) performed slightly worse in small sample sizes. Notably, these fit statistics also show different behaviors in small sample sizes than in large sample sizes, meaning that these model fit measures are prone to small-sample bias. There is an analytical derivation of removing the small sample bias from the SRMR (resulting in the unbiased SRMR; see [8]). However, the additional loss of information in planned missingness designs exaggerates the effect of the small-sample bias in the model fit statistics. I recommend considering resampling-based correction methods based on bias-corrected bootstrap or bias-corrected jackknife estimates of the fit statistics.

I would like to thank all the authors of the four articles of this Special Issue for their excellent contributions. Moreover, I would also like to sincerely thank all reviewers, handling editors, and the editorial staff of the Psych journal for their support.