Scoring Individual Moral Inclination for the CNI Test

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The article is presented in a clear way proposing the EIR tree model for the CNI test. The work is very interesting.

Author Response

Thank you so much for your review and recognition!

Reviewer 2 Report

Comments and Suggestions for Authors

The paper is well written and appears to offer an improvement in methodology over the existing methods. That said, my primary concern of this paper regards the Stats readership.

 

How many readers of Stats are familiar with the following terms? I use the word "appears" in a previous sentence because the analysis methods employed in the paper are not clear to me, and I would guess, not to the typical Stats reader. To publish this paper I would prefer to see the following terms more fully enunciated. Does the typical Stats reader understand EIRTrees?

95: deontological tendency

112: multinomial processing tree (MPT)

124: CAN algorithm

133: EIRTree (the implementation of which is a primary purpose of study).

215: The parameters of the EIRTree models were estimated with the Hamiltonian Monte 215 Carlo algorithm, which is a Markov Chain Monte Carlo (MCMC) method based on No- 216 U-Turn sampler (NUTS)

I am guessing, and it is a guess, that the typical Stats reader does not understand nor have a priori interest in these terms. As such, the paper in its current form would not present meaningful information to the reader, and so not be of much interest.

My suggestion, admittedly somewhat vague, is to provide more methodological context of these terms and goals of the paper. As indicated, the work does appear to be of good quality. Can the methods be more fully enunciated without adding too much length? For that, I do not have an answer. But if the language and theoretical models underlying this paper can be better and efficiently communicated, then perhaps there is a place for introducing the successful use of a new methodology for Stats readers.

Some further comments follow.

I am certainly familiar with traditional psychometric approaches to scale development, and wonder why these methods are not applied here. I suspect there is a good reason for this but would like to know why the EIRTree approach is superior. It may very well be in this context but as a reader I would like to understand the comparative advantages.

152: What are some example items? What do these items look like? Under what conditions does the respondent answer them?

256: Table 1 compares the two models based on LOO and WAIC by calculating their expected log pointwise predictive density (ELPD) for LOO or WAIC to determine their predictive accuracy.

312: A primary result:

"The results indicate that the EIRTree model with C→N→I structure fits data better than the model with N→C→I structure." Should spend some space addressing the meaning and implications of this result.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Stats is a Journal about statistics and data analysis. The paper begins with a discussion of moral dilemmas. The topic of moral dilemmas for this journal is subsidiary to the methodology.  From what I can tell, and without being expert in the authors specific form of analysis, there is useful information here. Further, a substantive topic such as moral dilemmas can play a viable role as an illustration that highlights a specific methodology. 

I had recommended the authors reframe their paper as such, not just adding a few paragraphs at the end, which I did find useful, but insufficient. I recommend to begin the paper with a discussion of the relevant methodology and compared to traditional psychometrics. and then illustrate the methodology with the substantive topic of moral dilemmas. Stats readers maybe be interested in the methodology but not specifically the topic of moral dilemmas. Without such a more in-depth revision, I would recommend publication of the paper in a journal that focuses on such substantive topics.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf