1. Introduction
Information and communication technologies (ICT) have transformed the practices of literacy, earning a great importance in the functioning of the contexts of the so-called Society of Knowledge. However, being immersed in a digital society does not ensure the same opportunities related to access and use for all citizens [
1]. Therefore, it is necessary for the population to master its languages and to possess a multiple literacy and communication competency.
The founding Treaty of the European Union ensures that “the Community will contribute with the development of high-quality education, fomenting the cooperation between Member States” [
2] (p. 66). In one of the recommendations given to the member states, the European Council of Lisbon proposed the creation of a new European framework of reference to “define the new basic qualifications that permanent learning should provide” [
3] (p. 11). This measure is considered essential for providing a response to the globalization phenomenon, transforming the socio-economic panorama with policies based on the Society of Knowledge. Along this line, the European Council of Stockholm recommends, as the main objective, an “improvement of the basic skills, especially in digital subjects and information technology” [
4] (p. 4). This priority also highlights the need to improve education policies based on the permanent competence building.
European policies are aimed at providing new basic skills, underlining the fundamental role of digital technology for the permanent training of the population [
5] and the importance of integrating it into the world of work [
6]. For this, the so-called points of reference are adopted, which are the basis for the development of key skills for life-long learning [
7].
The eight key competences for life-long learning are defined in the EU as “a combination of knowledge, skills and attitudes that are adequate to the context” [
8] (p. 7) and provide a common reference framework destined to the decision makers, educators, social partners and the students themselves. Ultimately, these are the set of knowledge, skills and strategies needed by a person to solve any obstacle found in the Society of Knowledge itself.
One of these key competences is the digital competence, understood as that which “involves the confident, critical and responsible use of the technologies from the society of information for work, entertainment and education” [
8] (p. 9). The ability to use technology to live, work and learn throughout life is considered a fundamental and transversal subject in light of the development of any community education program. From this point of view, the importance of taking advantage and integrating the technology in an efficient manner in the training/teaching centers should be highlighted. Likewise, the proposal is made for the development of international education plans that aim to develop the professional skills of the general population and the agents in charge of their training [
9]. Altogether, the European Strategy 2020 highlights the key enabling role that the ICT should play within the Society of Information [
10].
The above suggests that the existing relationship between the ICT and competency and education policies is becoming ever more important for a society found within the Society of Knowledge. For this reason, and beginning with the previous references, the need is justified to count with useful instruments that are current, valid and reliable for the study of digital competence. Furthermore, these tools must be adapted to the contexts where they are utilized [
11]. Thus, the objective of the present work is to validate the self-diagnosis questionnaire “DigCompEdu Check-In” in the higher education context.
2. Teachers Digital Competence
Education institutions are currently facing the challenge of seeking to innovate ways of conducting their education work, especially in light of the recent technological, economic and social changes that are currently taking place [
12], with the aim of training and educating new generations for an uncertain and disconcerting future.
Never in history has humanity had so many ICTs available as in the present. Technologies that are duplicated at great speeds thanks to digitalization, resulting in transmedia, i.e., the viewing of media content in different devices. This fact multiplies the manners in which people can interact with information [
13,
14]. This problem is in constant evolution, just as the development of technology [
15]. Thus, the development of competences should be perceived within the idea of lifelong learning.
Furthermore, the forecasts indicate that in the “Fourth Industrial Revolution”, within which we find ourselves in, most of the future jobs will require digital skills [
16]. For this reason, and without forgetting that this same technological penetration entails the so-called technological unemployment, the only way to overcome it is through the development of competences related with digital technology [
17,
18]. As a result, the development of digital skills, inherent to our society, are directly related with success, employability, creativity and the prosperity of each individual.
Within the professional competences of the teacher, different authors allude to those skills or abilities related with the use of technological tools for conducting their professional activity in the classroom, which are different in many aspects to the use that could be given to them in the household environment [
19]. The modern education institutions require the addition of technologies from the society in which they operate, which at the moment are digital in nature. This demands that the teachers possess a significant Teachers Digital Competence (TDC) for the mastery of the Information and Communication Technologies (ICT) and their integration into the teaching and learning processes [
20,
21]. This competence is understood as a set of capacities, abilities, knowledge or skills that teachers possess to solve educational problems by integrating ICT [
20]. At the same time, the mastery of the TDC empowers the teacher for the use of the ICT not only as support for their existing practices, but also to transform them [
22].
Some factors are presented as mobilizing variables of the TDC, among which we will discard some, especially having in mind their significance when we use them in the experimental part of the work:
Training of the teacher: the teacher’s work experience, initial training and the degree of knowledge of the ICT tools are a fundamental factor for the development of TDC [
23,
24,
25].
Resources: quality of the infrastructure and availability of the digital devices and technologies necessary [
26]. There are teachers who assert their pre-disposition for integrating ICT resources into the teaching-learning practices if they had the necessary means [
27].
Usage time: dedication to the use of the ICT in and out of the classroom as an element that favors the teacher’s digital competence. Likewise, the lack of time available to prepare the sessions through the use of technologies is an opposing element [
28].
Attitude towards technology: the attitudes and beliefs the teacher has with respect to the possibilities provided by the ICT for teaching are critical variables that will determine the addition of the ICT to the teaching practice of the teacher, and not only their addition but also the manner in which they are introduced and the functions assigned to them [
29,
30,
31,
32,
33]. This attitude is perceived in the teacher’s use of certain technologies such as social networks [
30,
34].
At the same time, the use of technologies in the classroom has been promoted by education policies and didactic and pedagogic trends [
35].
Ultimately, the TDC is related with all of the skills, attitudes and knowledge required by the teachers in a digital world [
36]. Altogether, it is related with the use of the ICT from a didactic-pedagogic perspective in a professional education context which has an effect on the teaching strategies that are directly or indirectly related with technology [
37]. Furthermore, it has become an important line of research within the field of Education Technology [
38].
For the acquisition of the teacher’s digital competence, a series of competence frameworks have been suggested at the institutional level from different institutions, on which teachers should be trained. All of these seek to discover in which manner the Technologies should be integrated and used in teaching, to identify the training needs and to propose personalize training itineraries [
39]. Furthermore, in agreement with comments by different authors [
40,
41,
42,
43,
44,
45], the following can be considered the most consolidated: the European Digital Competence Framework for Educators, DigCompEdu; ISTE standards for Educators; the UNESCO ICT Competency Framework for Teachers; the Spanish Common Framework of Teacher’s Digital Competence; the British Digital Teaching Professional Framework; the Colombian ICT Competencies for the professional development of teachers; the Chilean ICT Competencies and Standards for the teaching profession. Altogether, an evaluation study through the use of expert competence coefficient highlighted the DigCompEdu model as the most adequate for evaluating the Teachers Digital Competence [
42].
3. The DigCompEdu Framework
At the end of 2017, the Joint Research Centre (JRC) published the European Digital Competence Framework for Educators: DigCompEdu [
46].
The DigCompEdu framework is the result of different professional meetings, workshops, debates and deliberations with experts and professionals. It is based on an initial bibliographic review and it entails the synthesis of existing instruments at the local, national, European and international levels [
31,
46]. The result implies a consensus on the main areas and elements of digital competence of teachers, and follows a logical progression in each competency area.
It is model of digital competence for teachers. Its competence areas are:
Professional engagement: it is focused on the work environment of the teachers. The digital competence of the teachers is expressed in their ability to use the digital technologies to improve teaching, but also to professionally interact with peers, students, family and different agent from the education community. This area is also focused in digital continuous professional development.
Digital resources: related with the sources, creation and distribution of digital resources. One of the key competences that any teacher should develop is the identification of education resources. Also, the teachers should be able to modify, create and share them so that they fit with their objectives, students and style of teaching. At the same time, the teachers should be able to responsibly use and administer the digital content, respecting the guidelines of copyrights and protecting personal data.
Teaching and Learning: know how to design, plan and implement the use of digital technologies in the different stages of the process of teaching and learning. Also, a change of approaches and methodologies that are centered on the students is advocated.
Assessment: linked to the use of digital tools and strategies in the evaluation and improvement of the processes of teaching-learning. The digital technologies can improve the existing strategies of assessment and give way to new and improved evaluation methods.
Empowering learners: one of the key strengths of the digital technologies in education is their potential to promote the active participation of the students in the process of learning and their autonomy within it. Also, the digital technologies can be utilized to offer learning activities adapted to the level of competency for each student, their learning interests and needs.
Facilitating learners’ digital competence: to develop and facilitate digital competence of the students.
This proposal is found within the “DigComp Initiatives”, and contributes to the “Competences for Europe Agenda” of the European Commission [
8,
47] and the Initiative of the European Strategy 2020 on New Skills for New Jobs [
15,
48]. The DigCompEdu framework is also used as a reference in some programs from the Autonomous regions of Spain for the digitalization of Spanish education centers, such as the Andalusian project #PRODIG [
49]. At the same time, DigCompEdu is compatible with the Common Digital Competence Framework for Teachers, published by the INTEF, in its different dimensions and levels [
50,
51]. Finally, DigCompEdu shares characteristics with other frameworks used internationally.
Table 1 compares with other frameworks.
It can be seen how almost all the frameworks have levels of competence, those that contemplate the progressive development of the competences. In this case, all of these classify these levels into three intervals (basic, intermediate and advanced). Regarding the dimensions and competences, all the frameworks contemplate technological and pedagogical competences. Technological: those professional competences that every teacher must develop within an educational institution committed to the Knowledge Society. Pedagogical: those competences directly linked to the teaching, learning and development processes of student citizenship.
Along the line of similar studies, where questionnaire-type tools were developed, based on the self-perception of the teacher and utilizing a Likert-type scale [
48,
52], DigCompEdu is utilized as the basis for the development of an online self-perception tool named DigCompEdu Check-In, which is considered to be the main object of study in this article.
4. Methodology
This cross-sectional research study intends to calculate the reliability and validity of the questionnaire DigCompEdu Check-In in the University Context of Andalusia (Spain). For this, the known-groups method is employed, replicating the procedure utilized by Ghomi and Redecker, where it was validated if the tool is able or not to discriminate the variable “digital competence” and its areas of between groups that are known [
31].
4.1. Sample
The sample was composed of 2262 professors from different public Andalusian universities, of which 1026 (45.4%) were men and 1236 (54.6%) women, with most being 50–54 years old (37.3%) and 666 between the ages of 40 and 49 (29.4%).
Table 2 shows the data collected with respect to the University of origin, field of study and years of teaching experience.
Lastly, it should be indicated, as a descriptive characteristic of the sample, that most of the teachers polled had been using information and communication technologies as educational tools for quite some time: 594 (26.3%) for more than 20 years, 466 (20.6%) 15–19 years, 448 (19.8%) 20 or more years, 312 (13.8%, from 6–9 years, 202 (8.9%) from 1 to 3 years and 38 (1.7%) less than a year. Not many teachers pointed to not using the technologies, more specifically, 34 (1.5%). This is shown in
Table 3.
It should be indicated that a great majority of the professors indicated that programs dedicated to the ICT existed in their center, and that they participated in it (f = 1156, 51.1%). Only 22.2% (f = 502) did not participate in their center’s existing programs, and 604 (26.7%) pointed that their center did not participate in these activities.
As for their competence in the use of different technologies, the answers provided are shown in
Table 4.
It should be indicated that most of the professors who completed the questionnaire were curious about new digital applications, programs and resources; more specifically, 28.7% (f = 650) were in agreement with the statement, and 1248 (f = 55.2%) were in complete agreement, together comprising more than 80% of the distribution, as opposed to 82 (3.6%) who were in complete disagreement, and 44 (1.9%) who disagreed. Only 238 (10.5%) of the professors were neither in agreement or disagreement.
As for the social networks utilized by the professors,
Table 5 shows the results found.
In the questionnaire, the professors were asked two times to score their digital competence as an educator, by placing themselves in the following groups: newcomer, explorer, integrator, expert, leader and pioneer. The scoring was done before completing the questionnaire, and after it was finished.
Table 6 shows the results of the assessment performed by the professors before and after finishing.
4.2. Data Collection Instrument
The data collection instrument used was the DigCompEdu Check-In questionnaire [
41], the analysis instrument of the European Digital Competence Framework for Educators DigCompEdu. This competence framework was selected through expert judgement in a previous study [
42].
The instrument is composed by 22, which refer to the 6 competences analyzed in “DigCompEdu”. They are related to the areas of competence: (A) Professional Engagement (four items), (B) Digital Resources (three items), (C) Teaching and Learning (four items), (D) Assessment (three items), (E) Empowering Learners (three items) and (F) Facilitating Learners’ Digital Competence (five items).
Each of the items measures the different competences that shape the competence framework: A1. Organizational communication; A2 Professional collaboration; A3 Reflective practice; A4 Digital Continuous Professional Development; B1 Selecting digital resources; B2 Creating and modifying digital resources; B3 Managing, protecting and sharing digital resources; C1 Teaching; C2 Guidance; C3 Collaborative learning; C4 Self-regulated learning; D1 Assessment strategies; D2 Analyzing evidence; D3 Feedback and planning; E1 Accessibility and inclusion; E2 Differentiation and personalization; E3 Actively engaging learners; F1 Information and media literacy; F2 Digital communication and collabo-ration; F3 Digital content creation; F4 Responsible use; and F5 Digital problem solving.
In the questionnaire, as previously mentioned, the professors were asked to assess their digital competence before starting the questionnaire, into one of the following categories: newcomer explorer, integrator, leader or pioneer. This was to be done after completing the questionnaire as well. The meanings of each level were the following:
Newcomer (A1): little experience and contact with education technology. Needs continuous guidance to improve his or her digital competence level.
Explorer (A2): little contact with education technology. Has not developed specific strategies for including ICT in the classroom. Needs external guidance to improve his or her digital competence level for teachers.
Integrator (B1): experiments with education technology and reflects on its adequacy for different educational contexts.
Expert (B2): utilizes a wide range of education technologies critically and with confidence and creativity. Seeks the continued improvement of teaching practices.
Leader (C1): able to adapt his or her needs to different resources, strategies and knowledge within his or her reach. A source of inspiration for others.
Pioneer (C2): questions the contemporary digital and pedagogic practices, of which they are experts themselves. Lead innovation with ICT and are a role model to follow for other educators.
Another series of demographic questions were added to the questionnaire: gender, years of service, time dedicated to the use of technologies in class, digital tools used in class, civil digital competence of the teacher, participation in social networks, etc.
4.3. Procedure of Data Collection and Analysis
The administration of the instrument was conducted through the internet (EuSurvey). It was administered in the first months of year 2020 in the different Andalusian public universities that provide regulated in-person teaching. The anonymity of the data was ensured. In addition, all teachers accepted confidentiality clauses.
To verify the reliability and validity of the questionnaire, the following two procedures were followed: acquisition of the reliability index and validation of the questionnaire. For this, the procedure from Ghomi and Redecker, the authors who created “DigCompEdu”, was used when checking for reliability and validation [
31].
To verify the reliability of the questionnaire, the Omega coefficient from McDonald was used, as it possesses a number of advantages compared to Cronbach’s Alpha; its resistance with samples of heterogeneous populations does not need the requirement of compliance with the tau-equivalence and the lack of correlated errors [
44,
53,
54,
55,
56,
57].
In addition, to validate it, the known-groups method was used, which was applied for analyzing the validity of an instrument and to discriminate the existence of significant differences in a variable of interest among known groups [
38,
57,
58]; it is a method utilized in different projects as the validation test in different areas of knowledge [
31,
59,
60,
61].
For the statistical analysis, the following non-parametric tests were utilized: Kruskal–Wallis, Mann–Whitney U and Spearman’s rho [
62,
63]. These tests were used since it has been verified that the data is not normally distributed through the asymmetry and kurtosis study. The “Kolmogorov–Smirnov goodness-of-fit test” confirmed this check, with significance
p = 0.000 for all items (non-normal distribution).
4.4. Hypothesis
To establish the known groups, the following hypotheses were formulated from the sociodemographic items:
Hypothesis 1. The more time spent utilizing technology as an education tool, the greater is the digital competence of the professor, and therefore, the general result of the test as well.
Hypothesis 2. The more time dedicated for using the technology in class, the greatest the digital competency, and therefore the general result of the test.
Hypothesis 3. The professors who know how to use different technology resources (computer, tablet, smartphones and internet), possess a greater digital competence, and obtain a better result on the test.
Hypothesis 4. The professors who are curious about new applications, possess a greater digital competence, and obtain a higher score on the test.
Hypothesis 5. The professors who are committed with some ICT or digitalization program at their university, promoted by the administration, obtain better results on the test.
Hypothesis 6. The greater the self-perception of the digital competence of the professor be-fore completing the questionnaire, the greater the digital competence, and therefore, the gen-eral result of the test.
Hypothesis 7. The greater the self-perception of the teacher’s digital competence after finishing the questionnaire, the greater the digital teaching competence, and therefore, the general result of the test.
Hypothesis 8. The greater the number of social networks utilized by the teacher, of which he or she is a user, the greater the digital teaching competence, and therefore, the better the results from the test.
The hypothesis that will be formulated will follow the following structure:
- ⯀
Null hypothesis (H0): there are no significant differences between the different groups com-pared, with an alpha risk of rejecting the null hypothesis of 0.05, and the total scores and the general results obtained by the teacher in the instrument “DigCompEdu”.
- ⯀
Alternative hypothesis (H1): there are significant differences between the different groups compared, with an Alpha risk of 0.05, and the total scores and general results obtained by the educator in the instrument “DigCompEdu”.
5. Analysis and Results
The analysis starts with the results of reliability (
Table 7).
All these values, in agreement with O’Dwyer and Bernauer, have high levels of reliability, for the overall instrument, as well as for the different dimensions that comprise it [
64].
It should be pointed out that for the items that had different response options, the Kruskal–Wallis test was utilized, and for the dichotomous answers, the Mann–Whitney U test was used.
Table 8 shows the Kruskal–Wallis values and level of significance for rejecting the H0.
As for the values obtained in the Mann–Whitney U test, these are shown on
Table 9.
The results obtained with both tests allow us to reject the H0 in every case at significance level p ≤ 0.001. As a result, it can be concluded that the instrument is valid for discriminating the level of TDC between the different response groups provided in the different items.
In order to analyze if there was a relationship between the answers provided by the subjects, and the final score reached in the instrument, and referring to the comparison of the hypothesis formulated, the values found after the application of Spearman’s rho are presented in
Table 10.
The results allow us to point to three aspects:
- ⯀
The relationships found are all significant at p ≤ 0.001.
- ⯀
All the correlations are positive, and as result, it can be pointed out that when one of the variables increases, the other does as well in the same direction.
- ⯀
Except for in the variables “Usage time of the technologies as an education tool” and “I know how to use the different technological resources (computer, tablet, Smartphone and internet)”, in which the correlations are low, the rest of them could be considered moderate [
63].
6. Discussion and Conclusions
The research study brings light to a number of aspects. The first of them is that we find ourselves with the instrument constructed for the analysis of the “European Digital Competence Framework for Educators “DigCompEdu”, an instrument with high levels of reliability, as referring to its totality, as well as the different dimensions that comprise it. These values are similar to those found by its authors, Ghomi and Redecker with German professors [
31].
The strategy used, the known-groups method, has shown to be significant for analyzing the validity of the instrument. Specifically for this case, the instrument was shown to have sufficient robustness, to discriminate the subjects who are clearly differentiated according to variables related with technology.
With respect to the reliability and validity of the instrument “DigCompEdu”, these allow us to create scientific knowledge with a useful degree of exactness and certainty for the improvement of the quality of education in the university institutions. At the same time, the results allow to establish, as a function of it, teaching and counseling plans [
44,
65], devising education systems that could cater to the demands of the Society of Knowledge [
66,
67]. These findings support the need for studies and research related to this topic. In addition, according to the section “ICT and Sustainable Education”, this topic is integrated into its line of interest, which is related to the development of the digital competences required in the present-day.
As future lines of research, the replication of this study should be conducted in other university contexts in order to achieve broader levels of agreement on the reliability and validity of the instrument [
68]. At the same time, it would be advisable to work on the creation of new instruments with two directions:
Performing of tasks by the professors to be able to evaluate their mastery of the competences, or lack thereof, through self-perception [
48,
69].
Increase the number of questions that are utilized in the diagnosis instrument [
27].
Regardless of what was stated, the research points to the need to establish university teacher training plans in the three main fields, as suggested by the DigComEdu model, and which have been validated in the research carried out. First, in the professional commitment, that the teacher acquires knowledge on how to use technologies to interact professionally with peers, students, and different agents of the educational community. Secondly, in having a solid training to mobilize digital technologies in the teaching-learning processes; this implies knowing how to identify digital resources appropriate to the characteristics of the students, being able to create innovative training environments through the incorporation of digital technologies, mastering the rules of copyright and under which conditions they can be modified by the teacher, in addition to having a scientific basis for knowing how to apply digital technologies in evaluation processes. Finally, having adequate training to know how to empower students in the use of digital technologies and create training actions so that students acquire a digital competence appropriate to the needs of the Knowledge Society.