Assessing Religious Orientations: Replication and Validation of the Commitment-Reflectivity Circumplex (CRC) Model

Abstract

The Commitment-Reflectivity Circumplex (CRC) model is a structural model of religious orientation that was designed to help organize and clarify measurement of foundational aspect of religiousness. The current study successfully replicated the CRC model using multidimensional scaling, and further evaluated the reliability, structure, and validity of their measures in both a university student sample (Study 1) and a nationally representative sample (Study 2). All 10 subscales of the Circumplex Religious Orientation Inventory (CROI) demonstrated good reliability across both samples. A two-week test-retest of the CROI showed that the subscales are stable over time. A confirmatory factor analysis of the CROI in the representative adult sample demonstrated good model fit. Finally, the CROI’s validity was examined in relation to the Intrinsic, Extrinsic and Quest measures. Overall, the CROI appears to clarify much of the ambiguity inherent in the established scales by breaking down what were very broad orientations into very specific suborientations. The results suggest that the CRC model is applicable for diverse populations of adults. In addition, the CROI appears to be construct valid with good structural and psychometric properties across all 10 subscales.

1. Introduction

Religious orientation is a foundational aspect of faith that includes a myriad of ways for approaching or avoiding religion. Commitment is the most easily identifiable trait of religious orientation but the underpinnings of why the individual is committed often manifest in many different ways due to a variety of motivations. Thus, religious orientation is multidimensional in nature and can promote human flourishing as well as create distress in some forms (Hathaway 2016; Hill et al. 2000, 2012; Krauss and Hood 2013). For example, varying forms of religious orientations were a motivating force behind the acts of martyrdom and terrorism witnessed in the attacks of 11 September 2001, as well as catalyzing the various acts of heroism, charity, and compassion exhibited in response to the same tragedy (Beck and Jessup 2004).

Much research over the past 50 years has been dedicated to measuring concepts related to religious orientation. For example, Hill and Hood (1999) presented over 200 published measures on this variegated construct. However, notwithstanding the development of many new measures since the publication of Hill and Hood’s compilation, additional efforts to organize this complex field are needed. Given the rapid expansion and the breadth with which the field continues to expand, researchers are left to rely on heavily outdated and/or disparate measures of this important construct.

Two of the most influential theories that have attempted to define religious motivation on an individual basis have been Allport and Ross’ (1967) Intrinsic/Extrinsic (I/E) motivation and Batson (1976) Quest (Q) model. I/E and Q were some the first to ask the important question of why a person seeks out or shies away from religion; rather, asking about behaviors and ideologies of faith commitment (e.g., frequency of prayer, church attendance, and doctrine of belief). Although I/E, and Q have been often considered the gold standard regarding religious motivation, there has been much debate over what these established scales are truly measuring (Beck and Jessup 2004; Gorsuch and McPherson 1989; Gorsuch 1994; Hill et al. 2012; Kirkpatrick 1989; Kirkpatrick and Hood 1990; Krauss and Hood 2013). For example, evidence suggests that Q and E are positively correlated and have the potential to be somewhat aggregated constructs (Batson 1976; Beck et al. 2001; Beck and Jessup 2004; Watson et al. 1989). Similar relationships have also been found between I and E (Hunt and King 1971; Kirkpatrick and Hood 1990). In an effort to address the multidimensional complexity of religious orientation, Krauss and Hood (2013) developed the Commitment-Reflectivity Circumplex (CRC) model of religious orientation (see Figure 1).

1.1. Commitment-Reflectivity Circumplex (CRC) Model

The CRC model (Krauss and Hood 2013), was developed using over a dozen Romanian and American samples with over 400 of their own items and over 20 established measures. The CRC model posits that religious orientation measures differ primarily with regards to the amount of commitment and reflectivity that they capture. In other words, these measures differ in the degree to which they tap into dedication to a religious faith (committed vs. uncommitted) as well as the degree that belief systems are analyzable and open to questioning and growth (reflective vs. unreflective). Therefore, an orientation could be categorized into one of four quadrants: Committed/Reflective (e.g., life-long learners), Committed/Unreflective (e.g., dogmatic and economic models), Uncommitted/Reflective (e.g., religious and spiritual struggles, and doubt), and Uncommitted/Unreflective (e.g., socially obligated and communal benefits).

The four quadrants are comprised of similar familial measures. Within a family, the measures are seen as fairly generic and interchangeable. For example, Krauss and Hood (2013) found minimal empirical differences between 12 different measures of faith commitment. The vast majority of established measures seem to capture varying aspects of faith commitment, which is a part of the Committed/Unreflective quadrant in the CRC model (Krauss and Hood 2013). For example, these measures included: the frequency of prayer, frequency of worship attendances, the Intrinsic scale (Gorsuch and McPherson 1989), Extrinsic Personal (Gorsuch and McPherson 1989), Christian Orthodoxy (Hunsberger 1989), Christian Religious Internalization (Ryan et al. 1993), Literal Affirmation (Duriez et al. 2000), and self-reported religiousness.

Only a few established measures appeared to fall outside of the Committed/Unreflective quadrant, almost as if faith commitment functioned as the force of gravity in the field. Established measures that fall into the Uncommitted/Unreflective quadrant include Extrinsic-Social (Gorsuch and McPherson 1989). The Uncommitted/Reflective quadrant consisted of measures such as Quest (Batson and Schoenrade 1991a, 1991b) self-rated agnosticism, and Reductive Interpretation (Duriez et al. 2000). The Committed/Reflective quadrant corresponds closely to what Allport (1950) termed “Mature Religion,” but was only captured by a few established indices such as interest in world religions and Restorative Interpretation (Duriez et al. 2000).

In short, the CRC model (Krauss and Hood 2013) shows much promise in the ability to structure and simplify measurement selection as well as providing avenues to more comprehensive measure religious orientation across world religions and at all levels of religiosity. As such, both the CRC model and the Circumplex Religious Orientation Inventory (CROI) have been influential in applied settings, such with the U.S. military (Shirley et al. 2016) and with federal chaplains (Krauss 2017).

1.2. Overview of the Current Study

The current study addressed three aims. The primary aim was to test whether the CRC model replicates in both a diverse student sample and a large nationally representative sample. The second aim was to more fully examine the structure and psychometric properties of the CROI. The third aim was to examine the convergent validity with established measures of religious orientation, namely Intrinsic, Extrinsic, and Quest orientations (Batson and Schoenrade 1991a, 1991b; Gorsuch and McPherson 1989).

2. Study 1: University Student Sample

Study 1 examined whether the CRC model replicates in a diverse university student sample, assess the psychometric properties of the CROI, and to examine the convergent validity of the CROI with established measures of Intrinsic, Extrinsic and Quest orientations (Batson and Schoenrade 1991a, 1991b; Gorsuch and McPherson 1989).

2.1. Method

2.1.1. Participants

The sample contained 174 participants (133 females and 40 males) from Central Washington University located in the Western United States. Participants were recruited through an online bulletin board for voluntary participation in psychological research. The average age of the participants was 21.09 years (SD = 5.04). Over 20 different religious affiliations were identified and recoded into 10 groups based on the Pew Research Center’s findings on America’s religious landscape (Wormald 2015). However, SES data was obtained from the university Office of Institutional Effectiveness for the total population of all students who were enrolled in psychology classes during the same quarter. The racial demographics, socioeconomic status, and religious affiliations of this first group of participants are listed in

Table 1. Descriptive Statistics of Racial Demographics, Socioeconomic status, and Religious Affiliation of both studies.

Demographics	Study 1 University Sample		Study 2 Nationally Representative Sample		USA Population
Race	n	%	n	%	%
African American	7	4.0	53	9.4	12%
East Asian	14	8.0	19	3.4	5%
European American	109	62.6	366	64.9	64%
Hispanic	27	15.6	51	9.0	16%
Middle eastern	1	0.6	2	0.4	1%
Native American	1	0.6	15	2.7	1%
South Asian	3	1.7	4	0.7	1%
Other or mixed	12	6.9	54	9.6	-
Total	174	1.0	564	100	100%
SES Annual Income	n	%	n	%	%
Under $15,000	234	15.5	75	13.1	14%
$15,000–$25,000	129	8.6	66	11.7	12%
$25,000–$35,000	97	6.4	61	10.8	11%
$35,000–$50,000	151	10.0	77	13.7	14%
$50,000–$75,000	152	10.1	101	17.9	18%
$75,000–$100,000	119	7.9	66	11.7	11%
$100,000–$150,000	155	10.3	69	12.2	12%
$150,000–$200,000	81	5.4	27	4.8	5%
Over $200,000	57	3.8	23	4.1	4%
No Report	330	22.0	0	0.0	0%
Total	1505	1.0	565	100	101%
Religious Affiliation	n	%	n	%	%
Evangelical	61	35.1	211	37.4	25.4
Catholic	36	20.7	112	19.9	20.8
Secular/Unaffiliated	46	26.4	97	17.2	22.8
Protestant	17	9.8	59	10.5	14.7
Nontrinitarian	1	0.6	13	2.7	2.4
Islam	1	0.6	10	1.8	0.9
Buddhist	3	1.7	8	1.4	0.7
Hindu/Pantheist	4	2.3	11	1.9	0.7
Jewish	0	0.0	6	1.1	1.9
Various Others	5	2.7	37	6.5	9.7
Totals	174	100	564	100	100

Note. The quota sample collected by Qualtrics approximates the U.S. population in terms of race, gender, age, and Socioeconomic Status (SES). The SES data reported for student sample reflects the total population (N) of all students who were enrolled in psychology classes during the September to December 2015 data collection period.

.

2.1.2. Measures

The Intrinsic/Extrinsic-Revised scale (I/E-Revised; Gorsuch and McPherson 1989) contains 14 items that are scored on a five-point Likert scale ranging from strongly disagree to strongly agree. The I/E-Revised is comprised of three scales Intrinsic (I), Extrinsic-Personal (Ep), and Extrinsic-Social (Es). The internal consistency of I (α = 0.82), and Ep (α = 0.84) were good. The internal consistency for Es (α = 0.79) was acceptable. These findings align with past literature (Hill 1999).

The Quest scale (Q; Batson and Schoenrade 1991a, 1991b) contains 12 items scored on a nine-point Likert scale ranging from (1) strongly disagree to (9) strongly agree. The Q was designed to measure aspects of openness to change (e.g., “I am constantly questioning my religious beliefs”), doubting as positive (e.g., “It might be said that I value my religious doubts and uncertainties”), and existential questions (e.g., “My life experiences have led me to rethink my religious convictions”). The internal consistency of Q (α = 0.80) was good and in line with past literature (Burris 1999; Steger et al. 2010).

The CROI (Krauss and Hood 2013) contains 63 items scored on a five-point Likert scale ranging from strongly disagree to strongly agree. The CROI is comprised of ten subscales, which are listed in

Table 2. Definitions of the CROI Orientations.

Religious Orientation	Definition
Centrality	Centrality orientation is the degree to which religion is important and central to an individual’s life. (p. 60)
Personal	Personal orientation is an approach towards religion in order to gain comfort, protection, forgiveness, and help in general. (p. 61)
Gain	Gain orientation consists of the degree to which religion is approached as a method for gaining wealth, health, success, and other personal desires. (p. 65)
Punishment	Punishment orientation is an approach to religion that is heavily colored by a fear of God, by conscious attempts to avoid divine punishment, and a belief that negative events are controllable as well as meaningful. (p. 67)
Obligation	Obligation orientation is the amount to which a person feels social pressure to act or be religious. (p. 71)
Social	Social orientation is the degree to which a person involves themselves in religion in an effort to make or see friends and others as social acquaintances. (p. 69)
Doubt	Doubt orientation is the degree to which an individual enjoys and values their religious doubts, uncertainties and questions. (p. 73)
Tentativeness	Tentativeness orientation is the degree to which an individual is self-critical and uncertain of the objective validity of their beliefs about religion (Whether religious or nonreligious). (p. 75)
Dialog	Dialog orientation “…is the degree to which an individual is aware that their religion is affected by ‘the contradictions and tragedies of life.”‘ (p. 76)
Interest	Interest orientation is the amount to which an individual enjoys learning, reading and talking about religion and religious concepts. (pp. 77–78)

Note. Each of the above definitions have been quoted by Krauss and Hood (2013). Page numbers of each source are included in parentheses.

. See Appendix A for a list of the scale items. The internal consistency of each subscale ranges are shown in

Table 3. Overall Internal Consistency of the CROI.

Sub-Scale	Krauss and Hood (2013)	Study 1	Study 2
Personal	0.80	0.88	0.90
Centrality	0.91	0.95	0.95
Gain	0.72	0.76	0.87
Punishment	0.73	0.85	0.82
Obligation	0.78	0.85	0.74
Social	0.84	0.80	0.86
Doubt	0.77	0.80	0.83
Tentativeness	0.77	0.80	0.83
Dialog	0.83	0.87	0.86
Interest	0.84	0.93	0.90
Average CROI	0.80	0.85	0.86

Note. Above are the internal consistency coefficients for the CROI. The coefficients in the Krauss and Hood (2013) represent an average over four U.S. samples. Therefore, the average is based on six total U.S. samples.

.

2.1.3. Procedure

The survey was administered via Qualtrics. The order of the scales was randomized. This study was approved by the university’s Institutional Review Board.

2.2. Results

2.2.1. Multidimensional Scaling Analysis

Multidimensional scaling (MDS) was conducted on the student sample to determine whether the CRC model was replicable. In the current study, the hypothesized Commitment and Reflectivity dimensions were extracted and all four theorized quadrants were apparent. Thus, the general premises of the CRC model were replicated. The MDS Euclidean distance model of the CROI subscales is depicted visually in Figure 2.

To determine whether the dimensions themselves were similar, the loadings from the current sample were rotated to the coordinates that Krauss and Hood (2013) found. A congruence coefficient of 0.88 for the Reflectivity dimension suggested that the Reflectivity dimension was very similar (Lorenzo-Seva and ten Berge 2006) to the one originally reported in Krauss and Hood (2013). However, the Commitment dimension only had a congruence of 0.80, which suggested that the Commitment dimension in the current study was somewhat different than that reported in Krauss and Hood (2013). In keeping with these earlier results, the current study found that Centrality, Personal, and Interest were the orientations highest on the Commitment dimension. However, the low pole of the Commitment dimension appeared somewhat different. In the current study, Social and Tentativeness demonstrated the lowest scores on the Commitment dimension, whereas Krauss and Hood (2013) found Social and Obligation to be lowest on the Commitment dimension. Overall, the Committed/Reflective orientations appeared to be lower on Commitment than found in Krauss and Hood (2013).

2.2.2. Internal Consistency

The reliability coefficients for the student sample are found in Table 3. An alpha level over 0.80 is viewed as good and over 0.70 is satisfactory (Loewenthal 2001; Streiner 2003). The average Cronbach’s alpha coefficients of the CROI (α = 0.85), indicate consistent and good overall reliability. All subscales had reliabilities equal to or greater than 0.80 with the exception of the Gain subscale (α = 0.76). Nine out of ten subscales posted reliabilities above those reported in the original work (Krauss and Hood 2013).

2.2.3. Construct Validity

The correlations between the CROI, I/E-Revised, and Quest are shown in

Table 4. Correlations from both samples between the CROI subscales and established measures of I/E-Revised and Quest.

	Intrinsic				Extrinsic Personal				Extrinsic Social				Quest
	Study 1		Study 2		Study 1		Study 2		Study 1		Study 2		Study 1		Study 2
Personal	0.72	**	0.77	**	0.80	**	0.65	**	0.20	*	0.12	**	0.12		−0.11	**
Centrality	0.87	**	0.88	**	0.67	**	0.53	**	0.16	**	0.11	**	0.00		−0.19	**
Gain	0.53	**	0.58	**	0.63	**	0.60	**	0.25	**	0.22	**	0.14		−0.01
Punishment	0.26	**	0.36	**	0.39	**	0.42	**	0.21	**	0.22	**	0.20	**	0.08
Obligation	0.21	**	0.13	**	0.23	**	0.24	**	0.21	**	0.29	**	0.20	**	0.21	**
Social	0.04		0.03		0.05		0.22	**	0.64	**	0.78	**	0.06		0.29	**
Doubt	−0.47	**	−0.50	**	−0.35	**	−0.40	**	−0.13		−0.02		0.36	**	0.35	**
Tentativeness	−0.54	**	−0.51	**	−0.20	**	−0.14	**	−0.09		0.02		0.36	**	0.43	**
Dialog	−0.03		−0.02		0.13		0.08		0.07		0.13	**	0.56	**	0.46	**
Interest	0.48	**	0.48	**	0.25	**	0.28	**	0.08		0.11	*	0.15	*	0.18	**

Note. * denotes p values less than 0.05, and ** denotes p values less than 0.001.

. In the student sample, the I orientation showed strong positive correlations with Personal and Centrality but relatively moderate correlations with Gain and Interest. Additionally, the I orientation showed moderate negative correlation with Doubt and Tentativeness. The correlations between I and Punishment and Obligation were small. Similar patterns were revealed in the correlations with Ep. Ep showed strong positive correlations with Personal, Centrality, and Gain, followed by moderate positive correlations with Punishment and negative correlations with Doubt and Tentativeness. Other than a strong relationship with Social, Es was less related to the CROI than I and Ep, Finally, Quest orientation showed strongest correlation with Dialog, followed by moderate correlations with Doubt and Tentativeness.

3. Study 2

Study 2 was designed to examine whether the CRC model replicates in a nationally representative sample, further evaluate psychometric properties of the CROI, and to examine the convergent validity of the CROI with Intrinsic, Extrinsic and Quest orientations (Batson and Schoenrade 1991a, 1991b; Gorsuch and McPherson 1989).

3.1. Method

3.1.1. Participants

The sample, comprising 564 participants, was a nationally representative quota sample that was recruited through Qualtrics (2016). The sample was recruited to be representative based on age, gender, race, and socioeconomic status. Respondents included 564 adults with an average age of 38.65 years (SD = 13.23), of which 288 were female and 276 were male. Racial demographics, socioeconomic status (SES), and religious affiliation are listed in Table 1.

3.1.2. Measures

The measures were the same as in Study 1. Reliability of I was α = 0.84. Reliability of Ep was α = 0.76. Reliability of Es was α = 0.85. Reliability of the Q was α = 0.80.

3.1.3. Procedure

Similarly, this study followed the same procedure as Study 1 and was approved by the university’s Institutional Review Board.

3.2. Results

3.2.1. Multidimensional Scaling Analysis

MDS was conducted to determine whether the CRC model was replicable in this nationally representative sample. As in Study 1, Commitment and Reflectivity dimensions were extracted and all four theorized quadrants were apparent. Thus, the general premises of the CRC model were replicated. The MDS Euclidean distance model of the CROI is depicted visually in Figure 3.

3.2.2. Internal Consistency

The reliability coefficients for the nationally representative sample are found in Table 3. The average Cronbach’s alpha coefficients of the CROI (α = 0.86), indicates consistent and good overall reliability. In the current sample, all subscales were equal to or greater than 0.80 with the exception of the Obligation Subscale (α = 0.74). Nine out of ten subscales again outperformed the reliabilities originally reported by Krauss and Hood (2013).

3.2.3. Test-Retest Reliability

As described above, a subset of 100 participants from the sample opted to retake the CROI approximately two weeks following the initial administration. An intraclass correlation (ICC) based on the absolute agreement of the two time points was computed for each subscale, as shown in

Table 5. Test-retest Reliability of the CROI in Study 2.

Subscale	Intraclass Correlation
Personal	0.95
Centrality	0.96
Gain	0.91
Punishment	0.79
Obligation	0.73
Social	0.83
Doubt	0.90
Tentativeness	0.90
Dialog	0.78
Interest	0.91

Note. Average number of days between tests was M = 12.94, SD = 1.96.

. The average of these coefficients was ICC = 0.87, suggesting that participants were generally consistent in their responses to the subscales.

3.2.4. Confirmatory Factor Analysis

A confirmatory factor analysis was conducted on the nationally representative sample to analyze whether the structure of the CROI could be replicated using the full 10-factor model of the CROI. This was done to test the CROI’s structure in a diverse sample and because Krauss and Hood (2013) used a sample size below the recommended eight participants per item plus an additional 50 (Meyers et al. 2006). By this standard, the sample size for the CROI should exceed 550 participants, which was accomplished with the current sample.

Two models were analyzed and goodness of fit was assessed for each. The first model did not account for response sets, and the second model added two method factors to control for the presence of reverse-scored items.

In the first model, individual items were restricted to load on only one scale. The second model was the same as the first, but added two method factors (for positive and negative items). Standards for interpreting the fit indices were based on the recommendations made by Meyers et al. (2006) as follows: CFI ≥ 0.90, PCFI ≥ 0.50, and RMSEA ≤ 0.06.

The results of both CFA models demonstrated satisfactory fit. In the first CFA, the model fit was adequate, χ²(1845) = 5534.27, CFI = 0.83, PCFI = 0.79, RMSEA = 0.06. In the second CFA, the model produced an even better fit, χ²(1781) = 4048.08, CFI = 0.90, PCFI = 0.82, RMSEA = 0.04. The factor loadings for the second model are reported in

Table 6. Factor Loadings for the CFA Model 2.

Subscales	Factor Loading	Positive	Negative
Personal
item 1	−0.650		0.451
item 2	−0.801	0.415
item 3	−0.599		0.139
item 4	−0.619	0.504
item 5	−0.760	0.444
item 6	−0.395		0.485
Centrality
item 1	−0.720	0.462
item 2	−0.814	0.346
item 3	−0.812	0.392
item 4	−0.837		0.298
item 5	−0.741		0.321
item 6	−0.770		0.441
Gain
item 1	−0.579	0.520
item 2	−0.435	0.621
item 3	−0.657	0.537
item 4	−0.555		0.315
item 5	−0.323	0.565
item 6	−0.529		0.201
Punishment
item 1	0.592	0.480
item 2	0.431	0.480
item 3	0.606	0.431
item 4	0.625	0.334
item 5	0.665	0.363
item 6	0.255		0.486
Obligation
item 1	0.638	0.534
item 2	0.368	0.543
item 3	0.589	0.434
item 4	0.481	0.136
item 5	0.589		0.317
item 6	0.322		0.549
item 7	0.247		0.099
item 8	0.352		0.426
Social
item 1	0.708	0.500
item 2	0.698		0.419
item 3	0.370	0.655
item 4	0.581	0.553
item 5	0.724	0.483
item 6	0.680		0.276
Doubt
item 1	0.796	0.219
item 2	0.695	0.079
item 3	0.642		0.015
item 4	0.643		0.211
item 5	0.674	0.271
item 6	0.619		0.073
item 7	0.652	0.343
Tentativeness
item 1	0.556	0.202
item 2	0.731	0.183
item 3	0.716		0.291
item 4	0.775		0.285
item 5	0.576	0.336
item 6	0.812		0.303
Dialog
item 1	0.475	0.423
item 2	0.775		0.401
item 3	0.704		0.407
item 4	0.633		0.369
item 5	0.625	0.334
item 6	0.726		0.421
Interest
item 1	0.701		0.323
item 2	0.675	0.397
item 3	0.635	0.432
item 4	0.681		0.300
item 5	0.718	0.418
item 6	0.697		0.435

.

Some of the factor loadings for items in the Obligation subscale appear to be relatively low, which may indicate that the Obligation subscale is measuring multiple similar constructs. Additionally, some items that loaded poorly on the Punishment subscale. Eliminating or revising these items could improve model fit.

3.2.5. Construct Validity

Similar to the patterns revealed in Study 1, the I orientation showed strong positive correlations with Personal, Centrality, and Gain but relatively moderate correlations with Punishment and Interest. Additionally, the I orientation showed moderate negative correlation with Doubt and Tentativeness. The correlation between I and Obligation was small. Similar patterns were revealed in the correlations with Ep. Ep showed strong positive correlations with Personal, Centrality, and Gain, followed by moderate positive correlations with Punishment and a moderate negative correlation with Doubt. Es was strongly correlated with Social but otherwise had had smaller correlations with the CROI subscales than I and Ep. Finally, Quest orientation was most strongly correlated with Dialog, followed by moderate correlations with Tentativeness and Doubt. These correlations for the representative sample are depicted visually in Figure 4 and support Krauss and Hood’s (2013) contention that I and Ep are Committed/Unreflective orientations, Es is Uncommitted/Unreflective orientation, and Q is an Uncommitted/Reflective orientation.

4. General Discussion

On the whole, both samples replicated the main points of the CRC model as well as supporting the CROI as a reliable and valid measure of religious orientation. At present, this study is the only known replication of Krauss and Hood’s (2013) work on the foundational concept of how religious orientations relate to one another. The current study also is the first to examine the test-retest reliability of the CROI. In general, the results show that most of the subscales exhibited good test-retest reliability.

As predicted by the CRC model, Commitment and Reflectivity dimensions were extracted in both samples. All four theorized quadrants were apparent, and no orientations were outside of the hypothesized quadrant. Thus, the general premises of the CRC model were replicated.

Some differences in the Commitment dimension were found between our findings and those reported in Krauss and Hood (2013), specifically in what orientations most indicated low commitment. In general, the Uncommitted/Reflective orientations of Tentativeness, Doubt, and Dialog were less committed in the current studies than was found by Krauss and Hood (2013). These orientations were derived from the Quest scaleand gives a unique perspective as to what might be motivating an individual high in Quest.

We also replicated the Krauss and Hood (2013) finding that Interest captures a highly Committed/Reflective orientation. This is reminiscent of Allport’s (1950) influential idea of “mature religion,” lending further support to Krauss and Hood’s claim that the analysis of spiritual and religious beliefs appears to be different than doubt and uncertainty.

Overall, the CROI subscales exhibited good reliability, and were consistently more reliable in both samples than originally found by Krauss and Hood (2013). In the nationally representative sample, Obligation was the only subscale that exhibited satisfactory, but not good, reliability. Further research may be useful for examining whether Obligation is measuring multiple constructs - one being how much a respondent’s loved ones care that they go to religious services and the other being how much those loved ones’ opinions influence or pressure the individual into going.

As for the structure of the CROI, the results suggest that the CROI scales accurately represent the measured dimensions. In fact, the current study showed better fit than did Krauss and Hood (2013), which is unusual for replication studies. However, there is some room for improvement in the CROI, particularly with regard to the Obligation subscale.

The CROI subscales were strongly and appropriately correlated with the original measures of I, Ep, Es, and Q. The pattern of correlations was virtually identical to that found by Krauss and Hood (2013). These correlations are suggestive of the CROI’s construct validity. These correlations also demonstrate the sensitive nature of the CROI subscales, which can provide insight into the original I, Ep, Es, and Q scales.

The I scale had the strongest association with Centrality in both studies, suggesting that the two scales are likely measuring approximately the same thing. Additionally, the I scale had a strong correlation with Personal, moderate associations with Gain, Punishment, and Interest, but was negatively associated with Doubt and Tentativeness. The evidence gives merit to the theory that the I scale is measuring a commitment to faith (Centrality, Personal, and Gain) that tends slightly toward dogmatism with little regard for ambiguity (Doubt and Tentativeness). In short, the I scale seems to be measuring a faith commitment that tends to be somewhat unreflective.

In both samples, Ep showed the strongest association with Personal, followed by positive correlations with Gain, Centrality, and Punishment. Additionally, Ep showed small positive correlations with Interest, and Obligation. Social also demonstrated a small positive correlation, but this was only significant in the much larger, nationally representative sample. Conversely, Ep was negatively correlated with the Uncommitted/Reflective orientations of Doubt and Tentativeness. These findings were similar to findings reported by Krauss and Hood (2013). Krauss and Hood suggested that overall, Ep demonstrates a Committed/Unreflective position. With that said, the correlation between Ep and Interest in the present study suggests that Ep may retain some reflective identity. This description makes more sense because, in the CROI’s theoretical structure, Personal borders Interest, which is a Committed/Reflective orientation. Furthermore, previous evidence suggests that I and Ep tend to be positively correlated (Kirkpatrick and Hood 1990; Krauss and Hood 2013), and if I is positively correlated with Interest; it would make sense that Ep might be as well.

In both samples, Es was most strongly associated with Social followed by weaker correlations with Obligation, Punishment, and Gain. Interest, and Dialog also demonstrated small positive correlations, but these findings were only significant in the nationally representative sample. These findings support the theory that individuals who score high in Es may vary in levels of faith commitment, but are primarily unreflective in their reasons for joining a religious community and/or participating in religious activities (Krauss and Hood 2013).

Quest (Batson et al. 1993) was most strongly associated with Doubt, Tentativeness, and Dialog, which are the three CROI orientations in the Uncommitted/Reflective quadrant. Quest also had small positive correlation with Interest, Obligation, and Social, which are the orientations in both quadrants neighboring the Uncommitted/Reflective quadrant. This indicates that Quest is clearly part of the Uncommitted/Reflective quadrant.

Quest (Batson et al. 1993) was originally developed to measure the reflective portion of Allport’s (1950) concept of a mature religion, which combines faith commitment with the ability to deal with complex questions pertaining to morality and evil, viewing uncertainty as healthy, and remaining open to change. In short, Allport’s mature religion was a Committed/Reflective orientation, similar to the CROI’s Interest orientation. However, Quest demonstrated only a small relationship with Interest in both samples and the CROI scales that were derived from Quest loaded at the low pole of the Commitment dimension in both samples. Both of these findings suggests that Quest is not measuring pure reflectivity, but instead a kind of reflectivity that is pulling away from religion. This finding supports Krauss and Hood’s (2013) contention that the Quest scale appears to be measuring something closer to agnosticism and spiritual struggle, with no real interest in religious theory. For example, Quest has been shown to predict a lack of interest in religion in multiple cultures (Beck and Jessup 2004; Krauss and Hood 2013; Watson et al. 1989). Krauss and Hood (2013) also found that Quest had no relationship with interest in world religions in 5 Romanian samples and an inconsistent, but generally positive, relationship with interest in world religions in 3 US samples. In contrast, Krauss and Hood (2013) found that Interest was consistently related to an interest in religion and an interest in world religions in both the US and Romania. Interest was also the only religious orientation to predict lower ethnocentrism in both US and Romanian samples after controlling for right wing authoritarianism (Altemeyer and Hunsberger 1992), and social dominance orientation (Pratto et al. 1994). In short, these two types of reflectivity are very different and indicate different levels of commitment.

Although this study supports the adequacy, reliability, and validity of the CROI and the CRC model, future research should be focused on the further development of the CRC model and the CROI. It is typical for circumplex models to measure quadrants more equally than the CROI does. For example, additional orientations beyond Interest should be added to better measure the Committed/Reflective quadrant. With that said, the CRC model does largely represent the current field of measurement in this respect. Additionally, it is possible that the finding in the current studies would not replicate in all populations. Future research should be focused on examining the functionality of the CRC model across nonwestern cultures.

A short version of the CROI that measures the CRC model should also be established. In the meantime, Krauss and Hood’s (2013) recommendation to use the Centrality, Interest, Doubt, and Social subscales seems justified. For researchers interested in measuring just reflectivity, Doubt (7 items) and Interest (6 items) may suffice, and would be very close in length to the 12-item Quest scale (Batson et al. 1993).

Practitioners may find the CROI useful for gaining a greater understanding of how religion is practiced in their clients’ lives. This may lead to identifying potential orientations that may be associated with negative mental health issues and maladaptive behaviors. The measure could then be used as an evaluation metric to determine if an individual’s religious orientation can be changed through interventions. In conclusion, The CRC model and CROI have been helpful in highlighting how religious orientations relate and can be grouped into families. This approach has highlighted that previous measurement approaches have rarely captured a committed and reflective orientation. Allport (1950) laid out the benefits that he observed in churchgoers who are both committed and reflective. However, Allport’s attempts to measure this orientation with survey instruments (i.e., the Intrinsic Scale and the Four-Fold typology) were unsuccessful (Batson et al. 1993; Krauss and Hood 2013). Batson attempted to capture the reflectivity within Allport’s (1950) committed and reflective orientation with his Quest scale, but the Quest scale has been shown to a capture an uncommitted approach that shows a lack of interest in religion (Beck and Jessup 2004; Krauss and Hood 2013; Watson et al. 1989). In contrast, Krauss and Hood’s (2013) Interest scale within the CROI appears to capture a committed and reflective approach.

The data from this study support Krauss and Hood’s (2013) assertion that the orientations measured by the established scales of I/E-Revised and Quest have been incorporated into the CROI. The CROI appears to clarify much of the ambiguity inherent in the older scales by breaking down what were very broad orientations into very specific suborientations and integrating them in a much larger landscape. This landscape can be used to clarify and group additional established models of religious and spiritual orientations so that future researchers can better make decisions about measurement. The CRC model and the CROI can thereby help simplify the field, while also ensuring more comprehensive measurement.

5. Materials and Methods

5.1. Materials

For both studies, the Statistical Package for the Social Sciences (SPSS) was used for the all analysis. Additional packages beyond the basic SPSS program were needed for the multidimensional Scaling (Alscale) and the Confirmatory Factor Analysis (AMOS).

All raw data have been reviewed by the university’s IRB. The full raw data set for the representative sample has been made available for public view at the Inter-university Consortium for Political and Social Research (ICPSR). However, the IRB noted that since the publication may be required to identify the institution, the specifics regarding religious affiliation and ethnicity posed a risk that someone from the student sample could be identifiable. Therefore, identifiable demographics have been removed from the student sample. All the data is accessible through the URL, http://doi.org/10.3886/E100943V1.

5.2. Methods

5.2.1. Multidimensional Scaling Analysis

The reverse coded items of the CROI were rescored and subscale scores were computed for each of the 10 subscales. The multidimensional scaling analyses was conducted through the Multidimensional Scaling (ALSCAL) option in SPSS. The Multidimensional Scaling (ALSCAL) was selected in the dropdown analysis box, and then the 10 subscale scores of the CROI were moved into the variables window. Next the “create distances from data” was checked and the Euclidean distance was selected for the measure type. All of the plot options were selected and then the analysis was then run. This process was done for both the university sample and the nationally representative sample.

5.2.2. Internal Consistency

Cronbach Alpha was used for both studies to test the internal consistency of all 10 subscales.

5.2.3. Test-Retest Reliability

Roughly two weeks after the representative sample was first collected, Qualtrics panels contacted the participants, N = 564, through email and collected data from the first 100 responses for the second sample. An average of the items in each subscale were created for time 1 and time 2. Intraclass correlations were then used to analyze the test-retest reliability between each pair of means.

5.2.4. Confirmatory Factor Analysis

The analyses were performed using the SPSS package of AMOS. In the first model, covariances between all 10 latent factors were allowed using a double-headed arrow. In the second model, positive and negative method factors were added to account for differences in responses to positive and negative items. All positive and negative items were tied to their perspective method factor using a single-headed arrow. The raw data set, n = 564, was attached to amos and used for the analysis.