CASD-14: A Questionnaire on Civic Attitudes and Sustainable Development Values for Service-Learning in Early Adolescents

Abstract

The service-learning disciplines can offer a unique opportunity for the civic development of sustainable values, as there is a large body of research that links them to values and civic attitudes. Given the need to develop instruments to measure the impact of service-learning methodology in early adolescence, a questionnaire was developed. For this purpose, a group of experts in the field of service-learning proposed a battery of items measuring three different constructs, named as follows: solidarity, compromise–responsibility and respect–dialog. The factor structure was examined, as well as its invariance across sex. The questionnaire confirmed good psychometric properties. Furthermore, the results have shown optimal goodness-of-fit indices. Sex was a variable to consider, as the analysis of invariance did not reach the scalar level. Lastly, the CASD-14 showed optimal test–retest reliability for both subsamples after eight months. The greatest contribution of this study is that it provides a direction for the application and sustainable development of the current scale, which might be considered a valuable assessment tool for early adolescent students’ service-learning. More research is needed with regard to sex-related differences in early adolescence.

1. Introduction

Today’s society faces constant challenges, where certain forms of knowledge can become obsolete [1]. In this sense, more flexible forms of teaching have emerged that give relevance to knowledge related to praxis, such as service-learning (SL). This is a teaching innovation method, combining learning objectives and social service with the aim of improving the educational quality and future employability of its students [2], while allowing for the development of civil competences related to citizenship and social transformation. In liquid societies, this methodology contributes to the training of citizens who actively participate and take on rights and responsibilities [3,4]. Liquid society is a metaphorical term, which is commonly used by sociologists [3,5] to define a reality focused on transitory issues rather than permanent ones. More specifically, SL provides an implementation of multiple aspects of traditional teaching through real context [6], and it integrates theories of intelligence by developing cognitive, instrumental and attitudinal or motivational aspects [7]. Thus, these programs must have a design that guarantees equal benefits for all agents in the process [8]. The scientific literature has shown the positive effects of SL on participation in community service, as well as civic values [9,10], pro-social behavior [11,12], motivation [13,14] and both personal and social orientations [15].

Another point of interest is how real-world learning opportunities are offered to students. In other words, how to link knowledge to activities that promote social and environmental responsibility by bringing teachers into novel pedagogic practices. These real-world learning opportunities seem to be a suitable way for students to develop not only key competencies, but also civic values responding to the construction of a citizenship in which solidarity and justice predominate [4]. Furthermore, these values are directly linked to the social and solidarity economy, and more precisely, to sustainability actions [16]. Of note, sustainability change agents are presumed to be, by nature, active learning agents, which is also expected in the SL methodology. The added value of this educational methodology lies in community service as a meeting place for active citizenship, for both professionals and students at any stage of development. However, most of the research on the contribution of the curriculum to education for sustainability is focused on university or high school students. To our knowledge, the literature on previous stages is rather scarce.

As mentioned before, SL methodology is of great interest for certain stages of development, such as pre-adolescence. In this case, secondary school students face unique problems of early adolescence that characterize them, such as biological and social changes, as well as cognitive transformations that allow them to tackle intellectual tasks typical of an adult [17]. Let us remember that the beginning of this period is characterized by the appearance of specific operations, while what characterizes the end is the appearance of formal thought [18]. In this regard, some studies [19,20,21] found greater social responsibility (described as students’ concern for the well-being of others, and the need to help others) in high school students belonging to interactive or service-learning programs, as well as the supportive participation of their parents or other significant adults. Furthermore, longitudinal studies suggest that pre-adolescents who engage in high levels of pro-social behavior are less prone to committing crime as well as exhibiting antisocial behavior [22,23]. In addition, pre-adolescents can not only receive, but also give back to the community what they have received. This type of methodological innovation experience helps them build and reinforce their identity from a positive self-conception [24].

It is important to understand the importance of pro-social actions in a student’s overall formation and, therefore, within the school context. The school plays a fundamental role in the development of social responsibility, based on reflection and participation complemented by civic ethics [25]. In this context, we could frame social responsibility as a student’s intervention to respond to social problems in their immediate environment. The importance of this value in school education makes the student an agent of social change and is directly related to the concept of active citizenship [26]. Understanding and exercising social responsibility is a fundamental quality that students must develop to become competent citizens and actively participate in society. An educational environment that emphasizes caring for others and fosters mutual respect and cooperation provides a space that promotes the development of a greater awareness of social problems, and ultimately, of the society around us [21]. Therefore, it can be considered a unique opportunity for the development of any student. Furthermore, it is important to remember that future generations, as citizens, will have to face changes that will pose urgent social and environmental problems [27]. Thus, the development of skills within this field has been considered crucial for democratic decision-making [28]. Unsurprisingly, sustainable development goals (SDGs) have, in many ways, been a question of debate in terms of monitorization and assessment. More precisely, the role of human values has commonly been excluded in the literature as it is considered a subjective variable that is difficult to measure. However, promising literature has shed light on how human values can be conceived as measurable indicators related to sustainability [29]. In this way, SL might be of interest for developing human values underlying SDGs through a group approach. Therefore, using teaching methodologies that contribute to promoting values and civic attitudes might be of interest in developing cognitive and rationally defined modes for the principle of sustainable development, which are essential to facing this liquid society, typical of modernity. Moreover, the benefits of SL also transcend the conceptual elements of the curriculum, such that this pedagogy opens awareness through interventions that guarantee sustainability and integral ecology.

Given the need to develop instruments to measure the impact of the service-learning methodology in early-stage students, the main objective of this study was to develop a tool to examine civic attitudes and sustainable development values for service-learning in this population. The following research questions were tested to assess the reliability and the validity of the proposed questionnaire (CASD-14):

(i)

Research question 1: If the measures are reliable, Cronbach’s α will be greater than 0.60 for each associated subscale.

(ii)

Research question 2: If the measures are valid, subscales will distinguish meaningful categories by confirmatory factor analysis.

To do so, two phases were proposed, and after a qualitative step, consistency and internal structure were assessed. So far, this has been a challenge in Spain, particularly for this type of pre-adolescent population. Although it was not a specific aim of this study, possible differences according to sex were considered through an invariance analysis, given the variability of social behaviors in men and women according to the previous literature [30,31,32].

2. Method

2.1. Procedure and Development of the Instrument

The proposed questionnaire was developed through three different stages. The first stage, called preliminary, consisted of developing and assessing the content of the items. To do so, a group of experts with proven experience in both formal and informal education was selected, specifically in SL (university lecturers, the staff of organizations related to education and NGOs). In the second stage, the most relevant pedagogical categories related to the SL axiological perspective were analyzed. Several discussion groups were held that appropriately outlined the pedagogical concepts or categories essential for the study. The grouping categories in SL were defined as follows: solidarity, commitment–responsibility and respect–dialogue. Moreover, a bank of 26 items was obtained that contained several proposals for each concept, which stipulated 9 items for the first subscales, and 8 for the last one. Subsequently, the experts were asked again to select which items best represented each concept or category by rating them from 1 to 10. All items had a mean higher than 8. After several cycles of analysis, the item bank was obtained for the questionnaire of interest, as described in Appendix A. In order to analyze the structure of the factors, a sample of pre-adolescents was selected and randomly divided into two independent samples, controlling for the variable sex. The second sample was slightly larger, in order to make it easier to subsequently carry out multigroup analysis.

Approval was obtained from the university’s ethics committee concerning the study, thus ensuring that the principles of the Helsinki Declaration were followed. Students were contacted to participate in the study with parental or a guardian’s approval, as well as with prior informed consent.

2.2. Participants

Participants voluntarily collaborated and received no compensation. All of them were in compulsory secondary education and involved in a school network linked to the Scholas Citizenship program in the city of Valencia. All of them were Spanish native speakers. Moreover, most of the participants also had Spanish nationality, with a small percentage (<6%) from other nationalities. For ethical reasons, other personal data were not recruited. First, a sample of 199 students participated in the exploratory factor analysis, where 50.3% were men and 49.7% women, with a mean age of 13.99 and a standard deviation of 1.94. Second, a sample of 221 students participated in the confirmatory factor analysis (51.6% male and 48.4% female), with a mean age of 13.96 and a standard deviation of 1.42.

2.3. Design and Analysis

The first step was a qualitative approach to develop the content of the proposed questionnaire, while the second step was a cross-sectional study under an incidental sampling. The analyses were carried out through SPSS 22 (IBM Corp., NY, USA) and Amos 18.0 module. First, an exploratory factor analysis (EFA) was carried out through a Promax oblique rotation. After removing the factorial solution, the confirmatory factor analysis (CFA) was carried out, where the goodness-of-fit was assessed through the chi-square indices and their relationship between degrees of freedom, a comparative adjustment index, known as CFI, and the root mean square error of approximation (RMSEA). Finally, a multigroup analysis was performed between the groups of boys and girls, belonging to the CFA sample, to assess invariance. This is a hierarchical procedure, which consists of starting with a model without restrictions and adding restrictions successively. The logic of this procedure is to test the factor homogeneity structure between the groups. Thus, several authors [33,34] recommend the analysis of invariance in developing a psychometric test. Furthermore, the literature [35] highlights its relevance in the study of the parameters of different populations. Therefore, three participant sex invariance models were examined, testing for scalar, metric and configuration invariance.

3. Results

3.1. Exploratory and Confirmatory Factor Analysis

Concerning the exploratory factor analysis (EFA) in the first independent sample, the Bartlett sphericity test resulted in the value p < 0.001 with a chi-square value of 1233.03 (gl = 496) and a Kaiser–Meyer–Olkin (KMO) sample index value of 0.656.

After performing the scree test and a parallel analysis, shown in Figure 1, a three-factor solution was selected.

Table 1. Component loadings for a three-factor solution and uniqueness per item.

	F 1	F 2	F 3	Uniqueness
ITEM 01	0.087	−0.262	0.155	0.904
ITEM 02	0.210	0.155	0.229	0.818
ITEM 03	0.407	0.185	−0.080	0.762
ITEM 04	0.397	−0.104	−0.007	0.967
ITEM 05	0.210	0.075	−0.026	0.942
ITEM 06	0.259	−0.313	0.140	0.844
ITEM 07	0.081	−0.010	0.503	0.708
ITEM 08	0.495	0.090	−0.371	0.722
ITEM 09	0.139	0.458	0.179	0.677
ITEM 10	−0.108	0.052	0.750	0.496
ITEM 11	−0.073	0.059	0.758	0.467
ITEM 12	0.263	0.315	0.206	0.686
ITEM 13	0.445	0.090	−0.107	0.790
ITEM 14	0.304	−0.003	0.140	0.854
ITEM 15	0.042	0.356	0.025	0.859
ITEM 16	0.148	0.437	−0.173	0.724
ITEM 17	0.366	−0.137	−0.026	0.892
ITEM 18	0.415	0.058	0.069	0.778
ITEM 19	0.350	0.062	0.137	0.800
ITEM 20	0.318	0.007	−0.008	0.899
ITEM 21	0.394	0.125	0.147	0.725
ITEM 22	−0.216	0.678	0.165	0.614
ITEM 23	0.145	0.539	−0.022	0.631
ITEM 24	0.096	0.562	0.059	0.628
ITEM 25	0.346	−0.010	0.040	0.870
ITEM 26	0.322	0.117	−0.040	0.863

characterizes the load factor after a Promax rotation. Likewise, Table 1 shows the factor loads.

After studying internal consistency, which reached acceptable values, as shown in

Table 2. Mean of scores, standard deviation in parentheses by sex and internal consistency by factor.

	N	FACTOR 1 (Solidarity)	FACTOR 2 (Commitment–Responsibility)	FACTOR 3 (Dialogue–Respect)
Men	99	2.78 (0.92)	3.82 (0.65)	4.09 (0.79)
Women	100	3.00 (0.86)	4.01 (0.56)	4.34 (0.60)
Internal consistency	199	0.7	0.6	0.66

and Figure 2, the factors were conceptualized according to the constructs of solidarity, commitment–responsibility and dialogue–respect. The scores were slightly higher for the group of girls than the group of boys, as shown in this table, and although factor 1 did not present statistically significant differences, factors 2 and 3 did, through the Student’s t-test for independent samples (all p-values > 0.05). Before carrying out a multigroup analysis, the structure of the factorial solution was confirmed through a second independent sample of 221 students, where the sex variable was also controlled. The model presented an optimal fit, except for the CFI index which was acceptable: χ² = 137.89, p < 0.001 (gl = 74), χ²/gl = 1.86, CFI = 0.83 and RMSEA = 0.06. Table 2 depicts the reliability through Cronbach’s alpha.

3.2. Multigroup Analysis

Finally, a multigroup analysis was performed to determine any significant differences in the structural parameters between the groups in terms of metrics. As shown in

Table 3. Goodness-of-fit statistics for tests of invariance across sex: a summary.

Model	χ2	df	χ2/df	CFI	RMSEA
Model 1: Configural invariance (baseline model)	217.898	148	1.472	0.815	0.046
Model 2: Full metric invariance	252.633	159	1.589	0.752	0.052
Model 3: Full metric and scalar invariance	286.469	173	1.656	0.699	0.055

, it is clear that there are significant changes when comparing model 2 with the less restricted model or baseline.

3.3. Test–Retest after Eight Months of Follow-Up

Finally, in order to examine the test–retest reliability of the new instrument, it was assessed again eight months after the first measurement. During these eight months, participants took part in a SL methodology by the Scholas Citizenship program in the city of Valencia. As shown in

Table 4. Test–retest after eight months of follow-up in each subsample.

		Test			Retest
		F1	F2	F3	F1	F2	F3
N = 199	Solidarity (F1)	1	0.131	0.078	0.313 **	0.155 *	0.071
	Commitment–Responsibility (F2)		1	0.347 **	0.098	0.485 **	0.393 **
	Dialogue–Respect (F3)			1	0.060	0.464 **	0.559 **
N = 221	Solidarity (F1)	1	0.089	−0.031	0.437 **	−0.103	−0.148 *
	Commitment–Responsibility (F2)	0.089	1	0.038	0.062	0.383 **	0.021
	Dialogue–Respect (F3)	031	0.038	1	−0.061	−0.033	0.445 **

* p < 0.05; ** p < 0.01.

, the Pearson correlation coefficient showed high test–retest reliability for each factor in each subsample.

4. Discussion

These results are of interest both theoretically and for application. In this sense, let us remember that the literature shows that the theoretical and practical aspects involved in service-learning are capable of promoting social justice, democratic values and sustainability, particularly for the constructs under study. The literature seems to observe certain differences in relation to sex-dependent pro-social behavior in pre-adolescents [30,31,36]. For this reason, despite not being a specific aim of the study, this variable was controlled and included in subsequent analyses. Thus, it was not possible to achieve a scalar level of invariance. Therefore, this could suggest that, although the factorial structure is similar for the groups, there are differences in the overall behavior responding to the items, and it may be slightly higher in some of the groups. Given the sex-dependent developmental differences highlighted by the literature [37], it seems that more research is needed on the subject. Likewise, there are certain limitations to take into account in the current study, which mainly concern the participants for various reasons. For example, the sample is based on a self-reporting technique and the study is incidental or for convenience, which could compromise some aspects of the generalization of the results; however, this is a very common technique in the literature [37,38,39,40,41].

Future lines of research should address the developmental component of these results, from pre-adolescence to adulthood. This would provide a complete view of the developmental component, and ultimately, a view of all civic engagement to understand the experiences of students in service-learning until they reach adulthood. The current literature has also recognized that sociocognitive components, such as values, skills and knowledge, may influence our understanding of education and civic engagement [42]. Furthermore, introducing service-learning into the classroom can promote more positive attitudes about pressing social problems. Therefore, it is desirable that future research should explore not only how students can tackle social problems of interest, but also how service-learning influences attitudes [43].

5. Conclusions

The aim of this paper was to develop a measurement instrument that would make it possible to discover possible changes concerning the concept of civic value and student experiences in the pre-adolescent stage, which, moreover, follows a service-learning methodology. The three selected factors were confirmed as follows: factor 1, solidarity; factor 2, commitment–responsibility; factor 3, respect–dialogue. These types of assessment tools could help shed light on the usefulness of pedagogical methods and techniques that promote critical reflection, structured discussion and designing projects on the value of citizenship, responsibility and social justice. In other words, it is desirable to assess the contracts proposed through this questionnaire in order to guarantee the effectiveness of the teaching–learning processes and to promote the long-term commitment of the students. Moreover, the application is a type of tool which could, in turn, not only help us to discover the perceptions of service-learning students in order to consider the effectiveness of teaching–learning processes, but also to promote the development of civic habits projected into their local community, instead of civic actions that end in mere experience.