Next Article in Journal
“I Don’t Want to Spend the Rest of My Life Feeling Alone”: Postdoctoral Scholars’ Experiences of (Dis)Connection
Previous Article in Journal
An Illustrative Case Example in a K-12 School District: Considering Sustainable Mental Health Practices within Systematic, Equitable, and Collaborative Approaches
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Instrument to Evaluate Intercultural Competence in Pedagogy Students

by
Tricia Del Carmen Mardones
1,*,
Michelle Francois Paulet
2,
Juan Eduardo Ortiz
3,
Elisabet Díaz
4 and
Marcela Romero
5
1
Center for Advanced Research in Education, Institute for Advanced Studies in Education, University of Chile, Santiago 8320000, Chile
2
English Department, Faculty of History, Geography and Letters, Metropolitan University of Education Sciences, Ñuñoa 7750000, Chile
3
Department of Systemic and Social Pedagogy, Faculty of Education Sciences, Autonomous University of Barcelona, 08193 Bellaterra, Spain
4
Kindergarten Education Department, Faculty of Philosophy and Education, Metropolitan University of Education Sciences, Ñuñoa 7750000, Chile
5
Pedagogical Training Department, Faculty of Philosophy and Education, Metropolitan University of Education Sciences, Ñuñoa 7750000, Chile
*
Author to whom correspondence should be addressed.
Educ. Sci. 2024, 14(4), 381; https://doi.org/10.3390/educsci14040381
Submission received: 15 February 2024 / Revised: 19 March 2024 / Accepted: 25 March 2024 / Published: 6 April 2024

Abstract

:
Confronting the current educational reality involves developing intercultural competence during and after training to produce an impact on educational practices that promote positive classroom interactions with culturally and linguistically diverse students. In this context, this article presents the results of the construction and validation of the Intercultural Competence for Pedagogy Students (ICPE) instrument in a Chilean university. Taking into account the focus of interest, intercultural education in initial teacher training, the validation sample of the survey included 521 students in initial teacher training. This research aimed to establish the goodness of fit of the scale by analyzing the reliability and validity of the construct through confirmatory factor analysis. The scale presented adequate psychometric properties, contributing to the measurement of intercultural competence in initial teacher training to support attention to students in their cultural diversity.

1. Introduction

Globalization has intensified the communicative interaction between cultures, giving rise to the need in higher education institutions to develop the relevant skills to deal with it in students. One is intercultural competence (IC), conceptualized as a set of skills, attitudes, and behaviors necessary for positive interactions between culturally diverse speakers [1]. There are several relevant theoretical models of IQ. One model traces the evolution of speakers from an ethnocentric plane of communication (exclusive use of one’s own cultural frameworks to interpret other cultures) to an ethno-relative one (acceptance and adaptation of cultural diversity in a society) [2]. Another model emphasizes self-awareness as a fundamental component in interaction with other cultures [3]. A third model posits that IC is established as a mechanism of attitudes, knowledge, and skills to interact appropriately in intercultural situations [4].
The Pyramid Model of Intercultural Competence [5] considers the convergence of the dimensions presented in other theoretical models, such as attitude, knowledge, and ability. The latter model is shown as processual and dynamic, where the different stages of competition are communicated. The essential characteristic of this model is given by the pyramidal representation, meaning that the higher the degree of acquisition of the components of IC, the higher the degree of intercultural competence obtained.
The Pyramid Model of Intercultural Competence, at its base, is composed of required attitudes, such as respect (valuing other cultures), openness (learning about other cultures without judging them), curiosity, and discovery (tolerance towards ambiguity and uncertainty). The second rung of the pyramid is shared by knowledge and understanding in conjunction with skills. Knowledge is composed of elements: cultural self-awareness, in-depth understanding and knowledge of cultures, culture-specific information, and sociolinguistic awareness. The skills component integrates them: listening, observing, interpreting, analyzing, relating, and evaluating.
The third level of the pyramid corresponds to the desired internal outcome component. Its elements are adaptability (to communication behaviors as well as to new cultural environments), flexibility (using appropriate behavioral styles with a new cultural context), ethnorelative vision (understanding of culture from the specificity of each cultural context), and empathy (ability to identify with a cultural other). The desired external outcome component, which involves behavior and communication effectively and appropriately, is at the top of the pyramid.
In the international context, culture is considered in the European Framework of Reference for Language Learning, Teaching, and Assessment [6] as an essential element for learning a foreign or second language, with IQ being a theoretical construct used for these purposes. In intercultural education, international organizations define interculturality as the equal interaction of diverse cultures through dialogue and mutual respect [7]. From a critical perspective, interculturality is not accessible in education, but, rather, a colonizing, modern, white, and Western gaze is overcome. Intercultural education, then, would seek to approach cultural knowledge denied and subordinated by modernity/coloniality [8].
Following the previous line, Chilean public policies preferentially emphasize interculturality in education, focusing on indigenous peoples without recognizing other cultures in curricular and pedagogical spaces [9]. This has put even more pressure on multicultural coexistence in Chile, which has been deepened by the sustained increase in the arrival of the Latin American migrant population, estimated at 1,736,691 million at the end of 2022 [10]. This fact impacts the education system, generating new challenges for schools and teaching [11].
The migratory phenomenon has shown that opening borders affects cultural contact and communication, bringing with it effects on culture, manifested in social interactions where individuals or groups relate [12]. This interaction is also affected by linguistic diversity, and there is a need to deal with the different languages that interact in school classrooms [13]. This interaction in schools has increased in Chile, especially with the arrival of a population of Haitian origin that communicates in Creole as their mother tongue and not in Spanish, making it urgent to pay attention to the processes of linguistic integration in schools as a result of this new scenario [14].
It is also observed that cultural diversity predominates in the current era’s globalized historical, social, and cultural context. This is an age where communication technologies have globalized, benefiting the teaching of various languages [15]. There is, therefore, a paradigm shifts from a monolithic society to a society of cultural pluralism, which leads to the need to promote intercultural competence in teacher training. In this way, educational interactions with the immigrant population are favored [16]. However, interculturality is also part of the pedagogical challenges of the 21st century, so it is necessary to develop skills in teacher training that help the coexistence of relationships [17]. Consequently, it provokes the urgent requirement of pedagogical attention to culturally and linguistically diverse students in teacher training, which is relevant for integrating interculturality in a transversal way in the training curriculum in pedagogical careers [18].
However, in higher education, intercultural competence has only been studied in recent years, giving importance to its development. Numerous communication problems result from multicultural coexistence, where cultural and linguistic differences converge or different interpretations of the communicative behavior between interlocutors [19]. In this subject, studies on teaching second languages predominate in initial teacher education and teaching foreign languages [20,21,22,23,24,25]. There are also IC studies in professional health training [26,27]. As for the instruments that measure intercultural competence, the literature mentions, for example, a scale to measure intercultural empathy [28] or the measurement of adaptation to various cultural contexts [29]. In the Chilean case, based on the Pyramid Model of Intercultural Competence, a scale of cultural intelligence is translated, operationalized, and constructed to be used in pedagogy students from three Chilean regional universities [30].
Consequently, studies related to the teaching of a foreign language in contact with another culture predominate in the literature on intercultural competence in initial teacher education, showing few studies in the field of mother tongue teaching. In addition, there is a need for more instruments that measure the dimensions established by different theoretical models on intercultural competence that are valid and reliable [31], presenting a methodological gap regarding the matter. Thus, there is a need for instruments that measure this construct, guaranteeing the identification of the impact of teacher training.
Therefore, through this communication, the research team takes on the challenge of having valid instruments to measure the construct of intercultural competence. It is understood that, in the Chilean context, with the arrival of a migrant population, mainly from Latin America, teachers must possess intercultural competence to interact with culturally and linguistically diverse students. For this reason, this research seeks to construct and validate a scale of Intercultural Competence for Student Teachers (from now on, CIEP) in the context of a Chilean university. From this perspective, the research team wonders if it is possible to measure the perception of intercultural competence through this new Likert scale in pedagogy students, understanding that future teachers mediate the interaction between the different cultures present in the classroom through language.

2. Materials and Methods

This study uses the quantitative research method as a way to construct and validate an intercultural competence scale with a non-experimental and cross-sectional design in order to search, in a short period, for traits, factors, characteristics, and other important properties that affect the intercultural competence to be studied, given that the data are collected at a single point in time [32] using a questionnaire.

2.1. Participants

The study was carried out using a non-probabilistic convenience sample of university students (n = 521) from two pedagogical careers at a Chilean university (Pedagogy in English and Pedagogy in Basic Education), belonging to four branches of a private university with regional headquarters. In the capital of Santiago de Chile, it was applied in two offices: the San Miguel Campus ( n 1 = 69) and the Providencia Campus ( n 2 = 34). In Southern Chile, they were applied at the Talca ( n 3 = 297) and Temuco ( n 4 = 121) sites. The distribution of the sample according to sex corresponded to 450 women and 71 men. On average, the students evaluated were in semester 5.04 (SD = 2.4), with an average age of 22.2 years (SD = 3.37), with a minimum age of 18 years and a maximum of 40 years.

2.2. Instrument

The ICPE instrument was designed to contribute to the knowledge and measurement of intercultural competence in pedagogy students. This goal was established based on the Pyramid Model of Intercultural Competence [5]. Four of the five components proposed by the model were considered scale components. The fifth was discarded because it was a component that pointed to the performance of the competition and, therefore, required direct observation for its evaluation. The four components selected were as follows: Attitude, made up of 15 items and associated with three conceptual categories (respect, curiosity, and openness); Knowledge and Comprehension, composed of 15 items and associated with three conceptual categories (cultural self-awareness, deep cultural knowledge, and sociolinguistic awareness); Integrated Skills with 9 items and associated with the conceptual category (knowledge processing); and, finally, Desired Internal Results, made up of 15 items and associated with four conceptual categories (integrated with adaptability, ethnorelative perspective, empathy, and flexibility).

2.3. Procedure

The research was carried out in five phases. The first phase (a) corresponds to the revision of the existing theoretical models and instruments for collecting information on IQ, an issue incorporated in other publications. The subsequent phases are phases (b), the construction of sentences based on the dimensions of the selected and operationalized theoretical model, (c) validation of content by expert judgment, and (d) analysis of the validity of the theoretical model of the instrument.
In the stage of content validation by expert judgment, a Likert scale [33] was designed with 54 items in the format of a polyatomic response with three categories: Disagree, Agree, and Strongly Agree. The literature considers these categories sufficient and relevant [34]. The scale was subjected to content validation by four highly experienced experts who evaluated the suitability of the sentences based on their extensive experience and knowledge in the area. To this end, the inter-judge agreement calculated with Lawshe’s strategy was used, which proposes the evaluation of the relevance of each item through two categories: “essential” and “non-essential”. Although the criteria analyzed by the judges were wording, content, congruence, and relevance, only the latter was used to calculate the CVR. Subsequently, qualitative data (comments) and quantitative data (QOL) were triangulated for the definitive retention of items [35].
The questionnaire was administered to students in their classrooms, with prior authorization from the institution’s academic units, and it was approved by the university’s institutional ethics committee. Informed consent was obtained from all participants, informing them that their response was voluntary and respecting their rights to privacy.

2.4. Data Analysis

The confirmatory analysis of the data provided by the application of the scale sought to statistically establish the unidimensionality of the components of the construct to be measured through the factorial loads of each sentence with the WLSMV (Weighted Least Squares Mean and Variance-Adjusted) method, for ordinal rating scales [36] and, in turn, to investigate the relationship between the general construct and the components defined by the CFA to analyze the suitability of the model The comparative fit index (CFI), the Tucker–Lewis index (TLI), the normalized Chi-square index (χ2/d.f.), the goodness-of-fit index through the mean square error of approximation (RMSEA), the convergent validity of the components of the model through the extracted variance (ρvc), and, finally, the reliability through the Jöreskog rho index. The statistical processing of the data was carried out using the R project software, version 4.0.0., with Yve Rouessell’s Iavaan package.

3. Results

3.1. Instrument

The content analysis considered the level of agreement among the panelists through the CVR index—Lawshe’s new strategy for calculating CVR [37,38]. It also considered the expert opinions’ coding, considering each item’s relevance criterion [39]. Finally, the results obtained from the CVR and the judges’ comments were triangulated to eliminate the items that did not present convergence.
Once the initial items were defined, a confirmatory factor analysis (from now on, CFA) was developed to establish the validity of the scale construct. In the confirmatory analysis, the reliability of the internal consistency is checked by the Jöreskog coefficient [40], which integrates the errors into its calculation. The use of TFA is justified by the need to evaluate the fit of the data to a specific factor structure previously hypothesized [36,41]. For this purpose, the WLSMV (Weighted Least Squares) estimator was used. Adjusted mean and variance for categorical data fit non-normal distributions, such as that presented by the study data.
The factorial loads are observed to evaluate the proposed theoretical model, which must be significant according to the Wald test; the adjustment statistics of the comparative Bentler–Bonett index (CFI) and the Tucker–Lewis index (TLI) must be greater than 0.9, a criterion used by [42,43] and the mean square error of approximation (RMSEA) (acceptable if it is less than 0.08 and good if it is less than 0.05), the Joreskog index for the reliability of each verified construct (more significant than 0.7), and the extracted variance index (greater than 0.5) [44]. Using the extracted variance’s rho allows us to determine if the indicators measure the associated construct. The acceptance criterion for the index of variance extracted from a construct is defined as 0.5 as the minimum value, meaning that the construct shares more than half of its variance with its indicators [45].

3.2. Content Validation

Regarding the content validation of the items of the study in question, it is possible to affirm that the Content Validity Ratio (CVR) of the Lawshe-Adjusted Model index 125 out of 131, which exceeded the value of 0.71. On the other hand, the value of the CVI calculated for the complete instrument was 0.77, which is considered acceptable, for which the critical value of the CVI is 0.7. The result of this statistical analysis resulted in the elimination of six items. After Lawshe’s analysis, according to the analysis of the judge’s comments, the suggestions and observations of the content of each item [46] were coded. This qualitative analysis revealed some items as redundant or complementary, adjusting the instrument by eliminating 71 items. The information obtained from the data triangulation finally allowed us to maintain 54 items that made up the instrument applied to the CFA sample.

3.3. Results of the Instrument Applied

Below are the characteristics of the instrument applied and its components to be reviewed through the CFA (Table 1).
After the CFA, the attitude dimension, initially composed of 15 items, was reduced to 8 items after the factor contribution analysis. The items eliminated included the following: item 2, “It is legitimate to ignore people of different cultures, races, and nationalities than mine”, item 38, “I appreciate that people of different cultures, races, and nationalities than mine have different rights than I do”, and item 41 “I like to always be right when I am in a communicative situation with people from another culture”. The reduced items show an adequate fit according to the reported statistics (χ2 = 127.8, d.f. = 43, χ2/d.f. = 2.9; p-value = 0.0; TLI = 0.988, CFI = 0.981, extracted rho variance = 0.51, RMSEA = 0.062, rho_Jöreskog = 0.89) (Table 2).
After analyzing its factorial contribution, the Knowledge and Comprehension dimension, initially from 15 items, was reduced to 9 items. Some of the items eliminated were as follows: item 7, “I only know the cultural norms that I use to communicate”, item 8, “I avoid interpreting the intention of my listeners when communicating”, item 26, “I prefer to avoid knowing the meanings of the cultural characteristics of my listeners when communicating”, and item 27 “I always express myself in the same dialect that the other person expresses”.
The reduced construct shows an excellent fit to the established validity criteria (χ2 = 60.43, d.f. = 56, χ2/d.f. = 1.08; p value = 0.32; TLI = 0.99, CFI = 0.99, extracted rho variance = 0.32, RMSEA = 0.012, rho_Jöreskog = 0.80) (Table 3).
In the Skills dimension, five of the nine items were eliminated because they had low factor loads. Some of them were item 29, “I avoid eye contact with my listener when conversing”, item 48, “I pay attention only to information that interests me in the communication process”, and item 13, “I constantly consider information to be correct or incorrect. given to me by people from other cultures”. Consequently, the dimension comprised four of the nine it originally had.
The statistics generated from the reduction in items present adequate adjustments (χ2 = 21.87, d.f. = 17; χ2/d.f. = 1.29; p value = 0.19; TLI = 0.99, CFI = 0.99, extracted rho variance = 0.49, Jöreskog rho = 0.79; RMSEA = 0.023) (Table 4).
Finally, in the Desired Internal Results dimension, 4 of the 15 items were eliminated, including item 18, “I find it difficult to adapt to communicative contexts outside my culture” and item 35, “I have always been competent in a conversation with a person from another culture”, which have a low factor load, leaving the dimension composed of 11 items. The resulting construct presents optimal statistical fits (χ2 = 90.36; d.f. = 76; χ2/d.f. = 1.19; p-value = 0.12; TLI = 0.99; CFI = 0.99; extracted variance rho = 0.47, Jöreskog’s rho = 0.91, RMSEA = 0.019) (Table 5).
The TFA results at the global level show an adequate fit, where each of the dimensions presents factor scores greater than 0.3 for each of the four dimensions (χ2 = 1306.10, d.f. = 559; χ2/d.f. = 2.34; p-value = 0.00, TLI = 0.98, CFI = 0.98, extracted variance rho = 0.81, Jöreskog rho = 0.94, RMSEA = 0.051). Table 6 shows a summary of the indices proposed to assess the TFA.
As shown in Table 7, in order to analyze the complete model, the reliability of the constructs was calculated with two indices: the coefficient of determination of individual reliability (CFI) and the composite reliability (Jöreskog’s rho). As the table indicates, the individual reliability of each of the items is significant since the value is close to 1. Regarding composite reliability, the value of Jöreskog’s rho index (0.94) indicates that each set of items explains part of the variation in the concept of intercultural competence with the measure of error. Finally, convergent validity greater than 0.80 is considered very good, while RMSEA is acceptable when it varies between 0.05 and 0.08 [45].
Figure 1 shows a graphical representation of the proposed model for intercultural competence.
Finally, Figure 1 shows a graphical representation of the proposed model for intercultural competence where the factor loads of IC are expressed as a theoretical construct in the components of the initial theoretical model. In this sense, IQ is strongly explained by the components of Knowledge and Internal Results, followed by the attitudinal component and the Skills component, respectively. This seems to coincide with studies in the area where it is found that the greatest challenge for institutions that train professionals is to develop skills associated with IC, as opposed to learning some notions related to the understanding of IC, which is the first thing that students in vocational training reach [47].

4. Discussion

This study sought to communicate the process of construction and validation of a measurement instrument for intercultural competence in university teacher training based on the Pyramid Model of Intercultural Competence [5] for the design and construction of the items, unlike other assessment instruments that feature only some dimensions of theoretical models [28,29,30]. The instrument is a Likert scale that allows four dimensions of the chosen theoretical model to be measured, given that studies have pointed out that it is essential to integrate interculturality in higher education transversally [18]. Therefore, this instrument contributes to the purpose of the evaluation.
The instrument proved valid and reliable for the purposes for which it was created. It should be noted that the validation of the instrument’s content is used through the inter-judge agreement, calculated with Lawshe’s proposal, ending with the triangulation of this analysis with the categorization of the qualitative comments made by the judges on the instrument. According to Romero et al. [38], such decisions made it possible to gather a broader spectrum of information beyond pertinence and relevance to interpret the experts’ answers later and understand their frame of reference, theoretically and representationally.
In this sense, the values obtained from the CVI above 0.70 are consistent with studies that applied these indices in content validation, such as that of Zamora de Ortiz et al. [48]. Confirmatory factor analysis was used for the instrument’s construct validity, where the four theoretical dimensions of the model [5] included in this study were evaluated: Attitude, Knowledge and Understanding, Skills, and Desired Internal Outcomes.
The results show that when applying the CFA, the instrument fits the model hypothesized in the first instance [49]. The model validated in this research allows us to affirm the composition of the construct in four dimensions present in the Intercultural Competence for Pedagogy Students (ICPS) model. This fact corroborates the initial underlying structure ratified by analyzing the factor loads of the analyzed dimensions (between 0.84 and 0.94), with items that present a high factor load. The literature shows that applying the CFA to validate the construct with which the instrument was constructed is recurrently used in education [50,51,52].
This goodness of fit makes it possible to define an instrument with a consistent factorial structure and items that adequately measure intercultural competence in pedagogy students. Thus, the final instrument consisted of 32 items, eliminating all items representing the theoretical concept of openness. This fact evidences the appropriate fit, given that this dimension presented a higher factorial score of 0.3. It can be interpreted that the items related to this theoretical concept would not discriminate, probably due to the precarious teaching in university education about the intercultural pedagogical actions received by the members of the sample or as a result of assimilationist and monocultural models in the university curriculum, such as the Chilean national school curriculum [53].
The goodness of fit between the CFA-derived and hypothetical models may vary depending on the sample size [48]. The consulted literature indicates that a sample of 500 people is considered very good [54]. Therefore, the adequate goodness of fit presented by the instrument after applying the TFA can be interpreted based on the size of the study sample, which is 521 students. In other words, the sample size considered more than 500 people, which influenced the adequate goodness of fit of the instrument.
However, even though the sample is considered very good, the researchers note limitations in the methodological aspects. Regarding reliability, Jöreskog’s rho could have been complemented with the Test–Retest strategy, which this study did not consider as a methodological procedure due to the scope of the research objective. However, it is hoped to respond to these challenges in future studies by applying the instrument to students from other careers that could be included in a second stage.
Even with the above, the results of the construction and validation of the instrument allow us to affirm that the instrument is adequate to measure intercultural competence in students of initial teacher training in higher education. Its contribution lies in constructing and validating an instrument that contributes explicitly to teacher training, considering the diversity in the Chilean school classroom that teachers in training must face. In addition, it is observed that most instruments that assess intercultural competence are based on something other than the totality of the theoretical dimensions of the theoretical models on the subject. On the contrary, they include one or two theoretical dimensions for their understanding.
On the other hand, as a projection of the study, the instrument allows for exploration of the development of intercultural competence in initial teacher training. For example, the theoretical dimensions of the instrument can be worked on didactically in a transversal way in the students’ professional practices. In that learning space, like internships, preservice teachers find themselves in classrooms with culturally and linguistically diverse students. Therefore, using this instrument makes it possible to project teacher training in intercultural competence.

5. Conclusions

In the first place, the objective set in the research is considered to have been achieved because it validates the theoretical decision to incorporate four dimensions associated with the proposed Pyramid Model of Intercultural Competence and the subsequent construction of the instrument. This makes it possible for the components to explain the construct of intercultural competence, given that all dimensions have excellent goodness of fit, with significant reliability and perfect convergence.
It is also essential to have been able to design and validate a model to measure intercultural competence with these characteristics and with a sample considered to be very good. The instrument has the value of offering educational teacher training to measure intercultural competence through diagnostic strategies. At the same time, students can be pedagogically trained through methodological devices associated with the validated dimensions to serve culturally and linguistically diverse students in the Chilean context.
Consequently, the results of this research open up a series of future investigations that allow us to work with the validated dimensions to access genuine reflections that deconstruct beliefs and judgments and enable professional competencies that strengthen professional development and interculturality in the classroom, in teachers in training and also in active practice.

Author Contributions

Conceptualization, T.D.C.M., M.F.P. and J.E.O.; methodology, E.D. and T.D.C.M.; software, E.D.; validation, E.D. and T.D.C.M.; formal analysis, E.D., T.D.C.M. and M.R.; investigation, T.D.C.M., M.F.P. and J.E.O.; data curation, E.D. and T.D.C.M.; writing—original draft preparation, T.D.C.M., M.F.P. and E.D.; writing—review and editing, M.R.; visualization, T.D.C.M.; supervision T.D.C.M. All authors have read and agreed to the published version of the manuscript.

Funding

Funding for this study was provided by DIUMCE 2019 PIED-FID Project, Metropolitan University of Educational Sciences; DIP Project 58-2015, Autonomous University of Chile; ANID/PIA/ Basal Funds for Centers of Excellence FB0003.

Institutional Review Board Statement

The research was governed by the Declaration of Helsinki and was approved by the Ethics Committee of the Universidad Santiago de Chile ETHICAL REPORT N°: 324/2019 with approval date 11 July 2019.

Informed Consent Statement

Informed consents were applied to study participants in their entirety.

Data Availability Statement

Data are shown in the article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Deardorff, D. Handbook for the Development of Intercultural Competencies; Storytelling Circles; UNESCO: Paris, France, 2020. [Google Scholar]
  2. Bennett, M.J. Towards Ethnorelativism: A Developmentalist Model of Intercultural Sensitivity In Education for the Intercultural Experience, 2nd ed.; Paige, R.M., Ed.; Intercultural Press: Yarmouth, ME, USA, 1993; pp. 21–71. [Google Scholar]
  3. Gudykunst, W.B. Towards a Theory of Effective Interpersonal Intergroup Communication: An Anxiety/Uncertainty Management (AUM) Perspective. In Intercultural Communicative Competence; Wiseman, R.L., Koester, J., Eds.; Sage: Thousand Oaks, CA, USA, 1993; pp. 33–71. [Google Scholar]
  4. Byram, M. Teaching and Assessment of Intercultural Communicative Competence; Multilingual Matters Ltda: London, UK, 1997. [Google Scholar]
  5. Deardorff, D. The identification and assessment of intercultural competence as a result of the internationalization of students in institutions of higher education in the United States. J. Int. Educ. Stud. 2006, 10, 241–266. [Google Scholar] [CrossRef]
  6. Cervantes Institute. European Reference Framework for Language Learning, Teaching and Assessment; Cervantes Institute: Madrid, Spain, 2002. [Google Scholar]
  7. UNESCO. UNESCO Guidelines on Intercultural Education; UNESCO: Paris, France, 2006. [Google Scholar]
  8. Walsh, C. Critical Interculturality and (de)Coloniality; Editorial Abya Yala: Quito, Ecuador, 2012. [Google Scholar]
  9. Mardones, T. Intercultural education in the Chilean national curriculum. J. Educ. Intersect. 2017, 7, 69–84. [Google Scholar]
  10. Ministry of Social Development and Family. National Socioeconomic Characterization Survey (CASEN); Ministry of Social Development and Family: Santiago, Chile, 2022. [Google Scholar]
  11. Poblete, R. Working with diversity from the curriculum in schools with the presence of migrant children: A case study in schools in Santiago de Chile. Educ. Profiles. 2018, 40, 51–65. [Google Scholar] [CrossRef]
  12. Sanhueza, S.; Cardona, M.C. Evaluation of intercultural sensitivity in primary school students enrolled in culturally diverse classrooms. J. Educ. Res. 2009, 27, 247–262. [Google Scholar]
  13. Mena, E.M.; Díez, J.P. Didactics for Multilingualism in Teacher Education: An Empirical Study from the Practicum. Open Classr. 2015, 43, 94–101. [Google Scholar]
  14. Campos-Bustos, J. Haitian Students in Chile: Approaches to the Processes of Linguistic Integration in the Classroom. J. Educ. 2019, 43, 433–450. [Google Scholar] [CrossRef]
  15. Vine, A.; Ferreira, A. Improvement of communicative competence in Spanish as a foreign language through video communication. J. Theor. Appl. Linguist. 2012, 50, 139–160. [Google Scholar] [CrossRef]
  16. Usallán, L. Cultural Pluralism and the Political Management of Immigration in Chile: Absence of a Model? Polis 2015, 14, 277–302. [Google Scholar] [CrossRef]
  17. Vila, E.; Cortés, P.; Martín, V. Teacher training and the importance of intercultural and emotional competencies from an ethical perspective. Modulema 2017, 1, 5–20. [Google Scholar] [CrossRef]
  18. Ibáñez, N.; Figueroa, A.M.; Rodríguez, M.S.; Aros, A. Interculturality in Teacher Training: A Contribution from the Voices of People of Indigenous Peoples. Pedagog. Stud. 2018, 44, 225–239. [Google Scholar] [CrossRef]
  19. Fong, C.S.; De Witt, D. Development of intercultural communicative competence: Formative assessment tools for Mandarin as a foreign language. Malays. J. Learn. Instr. 2019, 16, 97–123. [Google Scholar]
  20. Bernal, A. Authentic materials and tasks as mediators to develop the intercultural competence of EFL students. HOW J. 2020, 27, 29–46. [Google Scholar] [CrossRef]
  21. Espinoza, E.; Herrera, L.A.; Castellano, J.M. The Intercultural Dimension in Teacher Training in Ecuador. Psychol. Soc. Educ. 2019, 11, 341–354. [Google Scholar]
  22. Tosuncuoglu, I. Awareness of the intercultural communicative competence of Turkish students and instructors at university level. Int. J. High. Educ. 2019, 8, 44–54. [Google Scholar] [CrossRef]
  23. Nur Gedik, B.; Perihan, S. Intercultural Language Teaching and Learning: Teachers’ Perspectives and Practices. Particip. Educ. Res. 2022, 9, 268–285. [Google Scholar] [CrossRef]
  24. Zeynep, Y.; Kemal Sinan, Ö. Developing Critical Intercultural Competence in Second Language Teacher Education. J. Educ. Teach. 2021, 8, 1022–1037. [Google Scholar]
  25. Pagés, J.; Vega, J.C.; Camacho, Y. Conceptual framework for developing intercultural competence in the teaching-learning process of English. Varona. Sci.-Methodol. J. 2019, 68, 1–4. [Google Scholar]
  26. Veliz, L.; Bianchetti, A.F.; Silva, M. Intercultural Competencies in Primary Health Care: A Challenge for Higher Education in Contexts of Cultural Diversity. Cad. Saúde Pública 2019, 35, e00120818. [Google Scholar] [CrossRef]
  27. Marek, E.; Nemeth, T. Intercultural competences in healthcare. Orv. Hetil. 2020, 161, 1322–1330. [Google Scholar] [CrossRef] [PubMed]
  28. González, H.; Álvarez-Castillo, J.; Fernández-Caminero, G. Development and validation of a scale to measure intercultural empathy. Relief 2015, 21, 1–18. [Google Scholar] [CrossRef]
  29. Castro, A. The Assessment of Cultural Competencies: Validation of the ICC Inventory. Interdisciplinary 2012, 29, 109–132. [Google Scholar]
  30. Sanhueza, S.; Patrick, P.; Hsu, C.; Dominguez, J.; Friz, M.; Quintriqueo, S. Intercultural Communicative Competencies in Initial Teacher Training: The Case of Three Regional Universities in Chile. Pedagog. Stud. 2016, 42, 183–200. [Google Scholar] [CrossRef]
  31. Garrote, M.; Fernández, M. Intercultural competence in teaching: Defining the intercultural profile of student teachers. Bellaterra J. Lang. Lit. Teach. Learn. 2016, 9, 41–58. [Google Scholar] [CrossRef]
  32. Buendía, L.; Colás, P.; Hernández-Pina, P. Research Methods in Educational Psychology; McGraw-Hill: Madrid, Spain, 1998. [Google Scholar]
  33. Matas, A. Design of the Likert scale format: A state of the art. Electron. J. Educ. Res. 2018, 20, 38–47. [Google Scholar] [CrossRef]
  34. Asún, R.; Zúñiga, C. Advantages of Polytomic Models of Item Response Theory in the Measurement of Social Attitudes: The Analysis of a Case. Psykhe 2008, 17, 103–115. [Google Scholar] [CrossRef]
  35. Rodríguez, C.; Pozo, T.; Gutiérrez, J. Analytical triangulation as a resource for the validation of recurrent survey studies and replication research in Higher Education. RELIEF. Electron. J. Educ. Res. Eval. 2006, 12, 289–305. [Google Scholar] [CrossRef]
  36. Díaz, E.; Fernández-Cano, A.; Faouzi, T.; Henríquez, C. Validation of the underlying construct in a scale to evaluate the impact of educational research on teaching practice by means of confirmatory factor analysis. J. Educ. Res. 2015, 33, 47–63. [Google Scholar] [CrossRef]
  37. Baghestani, A.R.; Ahmadi, F.; Tanha, A.; Meshkat, M. Bayesian Critical Values for Lawshe’s Content Validity Relation. Meas. Eval. Couns. Dev. 2019, 52, 69–73. [Google Scholar] [CrossRef]
  38. Romero Jeldres, M.; Díaz Costa, E.; Faouzi Nadim, T. A Review of Lawshe’s Method for Calculating Content Validity in the Social Sciences. Front. Educ. 2023, 8, 1271335. [Google Scholar] [CrossRef]
  39. Carrión, C.; Soler, M.; Aymerich, M. Analysis of the Content Validity of a Problem-Based Learning Assessment Questionnaire: A Qualitative Approach. Univ. Educ. 2015, 8, 13–22. [Google Scholar] [CrossRef]
  40. Roussel, P.; Durrieu, F.; Campoy, E. Structural Equation Methods: Research and Applications in Management; Economica: Paris, France, 2002. [Google Scholar]
  41. Urbano, A.; Iglesias, M.T.; Martínez, R. Design and validation of the Timeshare Scale in the Couple (TCP). Psychol. Soc. Educ. 2019, 11, 165–175. [Google Scholar]
  42. Martínez-Gómez, J.A.; Bolívar-Suárez, Y.; Yanez-Peñuñuri, L.Y.; Gaviria-Gómez, A.M. Validation of the Dating Violence Questionnaire (DVQ-R) for victims in Colombian and Mexican young adults. RELIEVE 2021, 27, 1–16. [Google Scholar] [CrossRef]
  43. McDonald, R.P.; Ho, M.-H.R. Principles and Practice in Structural Equation Analysis Reporting. Psychol. Methods 2002, 7, 64–82. [Google Scholar] [CrossRef] [PubMed]
  44. Moral, J. Confirmatory factor analysis In Estadística con SPSS y Método de la Investigación; Landero, R., González, M.T., Eds.; Trails: María Anaya, Mexico, 2006; pp. 445–528. [Google Scholar]
  45. Fornell, C.; Larcker, D.F. Evaluation of structural equation models with unobservable variables and measurement error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar] [CrossRef]
  46. Tilden, V.; Nelson, C.; May, B. Use of qualitative methods to improve the validity of content. Nurs. Res. 1990, 39, 172–175. [Google Scholar] [CrossRef] [PubMed]
  47. Klein, J.; Wikan, G. Teacher education and international practice programmes: Reflections on transformative learning and global citizenship. Teach. Teach. Educ. 2019, 79, 93–100. [Google Scholar] [CrossRef]
  48. Zamora-de-Ortiz, M.; Serrano, F.; Martínez, M. Content validity of the P-VIRC didactic model (ask-see, interpret, walk, tell) through expert judgment. Univ. Educ. 2020, 13, 43–54. [Google Scholar] [CrossRef]
  49. Bazán, A.; Barrera, D.; Vega, N. Validation of constructs of reading skills and text production at the beginning of the generalization of the Reform in Mexican primary school. REICE Ibero-Am. J. Qual. Eff. Chang. Educ. 2013, 11, 61–76. [Google Scholar]
  50. Martín-Pastor, E.; Sánchez-Barbero, B.; Corrochano, D.; Gómez-Gonçalves, A. What competencies and skills identify a good teacher? Design of an instrument to measure the perceptions of preservice teachers. Educ. Sci. 2023, 13, 789. [Google Scholar] [CrossRef]
  51. Granero Gallegos, A.; Baena-Extremera, A.; Escaravajal, J.C.; Baños, R. Validation of the Academic Self-Concept Scale in the Spanish University Context. Educ. Sci. 2021, 11, 653. [Google Scholar] [CrossRef]
  52. Sánchez Fuentes, S.; Jiménez Hernández, D.; Sancho Requena, P.; Moreno-Medina, I. Validation of an Instrument to Measure Teachers’ Perceptions of Universal Design for Learning. Lat. Am. J. Incl. Educ. 2019, 13, 89–103. [Google Scholar] [CrossRef]
  53. Mardones, T. Intercultural Education: Constructions of National Identity and Otherness Present in the Discourse of Chile’s Official Curriculum. Braz. J. Dev. 2020, 6, 61270–61277. [Google Scholar] [CrossRef]
  54. Kazykhankyzy, L.; Alag&#xF6, N. To develop and validate a scale to measure the intercultural communicative competence of Turkish and Kazakh ELT trainee teachers. Int. J. Instr. 2019, 12, 931–946. [Google Scholar] [CrossRef]
Figure 1. Proposed measurement model for IC.
Figure 1. Proposed measurement model for IC.
Education 14 00381 g001
Table 1. Theoretical dimensions, conceptual categories, and items retained by content analysis.
Table 1. Theoretical dimensions, conceptual categories, and items retained by content analysis.
Unit of AnalysisTheoretical DimensionsTheoretical Concepts
Item Producers
Article
Intercultural Competence (IC)AttitudeRespect1-19-37-2-20-38
Curiosity3-21-39-4-22-40
Aperture5-23-41
Knowledge and UnderstandingCultural Self-Awareness6-24-42
Deep Cultural Knowledge7-25-43-8-26-44
Sociolinguistic awareness9-27-45-10-28-46
SkillsKnowledge Processing47-29-11-12-48-30-13-31-49
Desired Internal OutcomesAdaptability14-32-50
Ethnorelative perspective15-33-51
Empathy16-34-52
Flexibility17-35-53-18-54-36
Table 2. Retained elements of the attitude construct.
Table 2. Retained elements of the attitude construct.
P 01: I show an interest in other cultures.
Q 19: I listen carefully to people with a different culture than I do.
P 37: I value a different culture than the one I own.
Q 20: I value people of different cultures, races, and nationalities than my own.
P 21: I react acceptably to communicative situations with people from other cultures.
P 39: I like to participate in different communicative situations with people from other cultures.
Q 04: I seek out information when I get to know a new culture.
Q 22: I always want to know about cultures other than my own.
Table 3. Retained elements of the construct Knowledge and Comprehension.
Table 3. Retained elements of the construct Knowledge and Comprehension.
P 06: I know my religious, values, and linguistic beliefs when I communicate with people from other cultures.
P 42: I value the religious, evaluative and linguistic beliefs that appear during communication.
Q 25: I take into account the cultural norms of my recipient when communicating.
P 43: All communication has cultural norms.
Q 44: I can interpret my recipient’s intent according to their culture.
Q 45: I can always understand the dialect of my receiver.
Q 10: I always adapt to the linguistic norm of my language.
Q 28: I know the linguistic rules of my language.
Q 46: I like to respect the linguistic rules of my language.
Table 4. Retained elements of the Skills construction.
Table 4. Retained elements of the Skills construction.
Q 47: I like to maintain eye contact with my listener.
Q 11: When I communicate, I make eye contact with my listener.
Q 12: I pay attention when people from other cultures talk to me.
P 30: I value listening carefully to the receiver in the communicative process.
Table 5. Retained construction elements Desired Internal Results.
Table 5. Retained construction elements Desired Internal Results.
Q 14: I adapt to the cultural aspects of my recipient.
P 32: I always have to adapt to the target audience from another culture.
Q 50: Adapting to a receiver from another culture is suitable for effective communication.
P 33: I think the existence of different cultures enriches people’s worldviews.
Q 51: I believe that the cultural expressions of different cultures are not limited to a single territory.
Q 16: I’m attentive to responding to the needs of others in conversation.
Q 34: I understand the cultural needs of others in the conversation.
P 52: In a conversation, it is necessary to understand and respond to the needs of the recipient.
Q 53: I always try to be effective when conversing with someone from another culture.
P 54: I always manage to adapt to communicative contexts that are foreign to my culture.
P 36: It is important to adapt to the linguistic norms of other cultures.
Table 6. Summary of consistency and statistical alignment.
Table 6. Summary of consistency and statistical alignment.
Dimensionχ2/d.f.TLICFIRMSEA ρ vc ρ Jor
Attitude2.90.990.980.0290.510.89
Knowledge and Understanding1.080.990.990.0120.320.80
Skills1.290.990.990.0230.490.79
Desired Internal Outcomes20.130.930.940.0500.470.91
Instrument IC2.340.980.980.0510.810.94
Table 7. Summary of the proposed model.
Table 7. Summary of the proposed model.
Proposed modelχ2/d.f.RMSEACFITLI ρ vc ρ Jor
2.340.0510.980.980.810.94
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Mardones, T.D.C.; Paulet, M.F.; Ortiz, J.E.; Díaz, E.; Romero, M. Instrument to Evaluate Intercultural Competence in Pedagogy Students. Educ. Sci. 2024, 14, 381. https://doi.org/10.3390/educsci14040381

AMA Style

Mardones TDC, Paulet MF, Ortiz JE, Díaz E, Romero M. Instrument to Evaluate Intercultural Competence in Pedagogy Students. Education Sciences. 2024; 14(4):381. https://doi.org/10.3390/educsci14040381

Chicago/Turabian Style

Mardones, Tricia Del Carmen, Michelle Francois Paulet, Juan Eduardo Ortiz, Elisabet Díaz, and Marcela Romero. 2024. "Instrument to Evaluate Intercultural Competence in Pedagogy Students" Education Sciences 14, no. 4: 381. https://doi.org/10.3390/educsci14040381

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop