Adaptive Learning Supported by Learning Analytics for Student Teachers’ Personalized Training during in-School Practices

Abstract

This paper presents the results of the second phase of the international project “Improving Educational Innovation, Competitiveness, and Quality of Higher Education through Collaboration between University and Companies (EKT)”. The use of adaptive learning supported by learning analytics is proposed as a pedagogical strategy to work on the collaborative and personalized learning process that takes place during the school placement period of initial teacher education. Learning analytics is expected to facilitate the analysis of the different sources of information and data generated in the learning process. The collected data will be centralized in a learning record store (LRS), which will serve as a repository for xAPI compatible traces from the tools that make up EKT intelligent system. The system is expected to provide a strong support to decision-making so that participant agents can collaborate, advise, and contribute to the future teacher’s personalized training according to his or her progress and the context in which the practice takes place. The need analysis of tutors in the five pilot countries is presented, which has made it possible to define the process variables that make up the learning analysis architecture of the EKT system.

1. Introduction

Data, its processing, and the impact that this processing generates on human activities have become, in recent years, the focus of numerous investigations from all areas of knowledge [1], including education. Indeed, data mining techniques are becoming increasingly important in the education field [2], and there is consensus that learning analytics is the third wave of educational technology, becoming a new and promising field of study [3].

Learning analytics is defined as the measurement, collection, and presentation of data related to learners and learning environments in order to understand and improve learning and the learning environment [4]. The use of this technology helps to closely monitor learning, predict student performance, detect undesirable learning behaviors and emotional states, and identify at-risk students for quick follow-up action [5], thus facilitating a learner-centered approach.

The literature reports that, although there is little transfer so far [6], the benefits of learning analytics are evident for institutions, which can use the available data effectively in decision-making [5]; for teachers, as they provide validated information for effective monitoring of student learning; and for students, enabling their learning experiences to be more personal and engaging, and also promoting reflection and improvement [5].

Recent developments in analytics usage can provide meaningful feedback on learning processes, which facilitates and enhances the success of learning processes [7]. The use of learning analytics is relevant in education, but, above all, it has a strong impact on higher education. It is especially useful in multi-stakeholder applications, to track learners’ progress, make performance predictions, suggest learning resources, detect undesired learning behavior [4], and evaluate.

Having listed some of the possibilities of learning analytics, it is necessary to point out that educational institutions are implementing interventions based on this technology, with little evidence on their effectiveness [7] and on the real impact of their transfer [6,8]. In fact, the literature review confirms that learning analytics is used to analyze data, i.e., to measure different parts of self-regulated learning, and not to apply learning analytics to support learners’ self-regulated learning [8].

The research confirms three fundamental critiques of the integration of learning analytics: it has been developed without the involvement of students and teachers; its use has been limited to identifying at-risk students; and institutions’ focus on grades, persistence, and non-completion metrics has been prioritized over motivation, engagement, student satisfaction, and more formative assessments of learning [9].

This fact suggests that it is necessary to consider, on the one hand, the needs and preferences of students, teachers, and experts, and, on the other hand, to focus on learning and on the people who will directly benefit from the use of learning analytics, i.e., teachers and students. Consequently, it is essential to involve teachers and students in the design of proposals based on this technology. Moreover, teachers will have to be aware of its potential and how they can integrate these data into teaching processes, and students will have to know how to exploit its possibilities to reinforce their autonomous learning and their decision-making capacity in the creation of routes for personalized learning. Indeed, to create effective interventions, higher education institutions need to understand how students interact with learning analytics tools and what factors influence those interaction patterns [10].

This confirms the need for more research on the design, implementation, and evaluation of learning analytics to build a solid evidence base on the feasibility, effectiveness, and generalizability of such interventions [7]. In this process, it is critical to consider pedagogical approaches and educational implications [11], focus research more on learning than on analytics [9], and attend to the implementation and evaluation of learning analytics [12].

Although research on learning analytics is prolific, only a small representation of these studies [13,14] have focused on its impact on future teacher education. In this paper, we present a set of specifications and functionalities that should guide the design and development of the e-learning system to support the practice of future teachers in schools.

2. Materials and Methods

2.1. Contextualization

This paper is possible thanks to the work done in the Erasmus project: Educational Knowledge Transfer (EKT). The EKT Project (reference number 612414-EPP-1-2019-1-ES-EPPKA2-KA) is a competitive research and development project coordinated by the Educational Technology Group of the University of Santiago de Compostela (Spain) and funded by the European Commission in the framework of the Knowledge Alliances call (Call EAC/A03/2018). It has a duration of three and a half years (2019–2023).

The EKT project aims to improve European higher education and, in particular, the development and experimentation of the collaborative educational methodologies and Smart technological framework, which aim to have an impact on a more effective, scalable, sustainable system which is closely linked to the professional contexts for which we train. To do this, we rely on a strategic triangle configured by educational centers (schools and teachers), universities (Faculties of Education and researchers), and technology-based companies and technological institutions (specifically those that offer services and e-learning resources). In this project, we work in an articulated way from these three contexts to establish knowledge development based on teaching innovation, educational research, and technological development.

The project is contextualized in the Faculties of Educational Sciences and in the official studies of Initial Teacher Education (ITE) in kindergarten, and primary and secondary education. With the implementation of the European Higher Education Area, ITE in Europe converges in the incorporation of a significant period of school teaching practice (in-school placement) that, in some cases, constitutes up to 25% of the prospective teacher training. The EU Commission, in its report “The Teaching Profession in Europe” (European Commission/EACEA/Eurydice,2015), highlights “real classroom experience” as one of the three key elements of ITE, emphasizing the importance of “in-school placement” at the same level of the “academic knowledge of their subject” and “teaching approaches”. Proper teacher training requires being trained in these three elements. Surprisingly, academic policies do not generally value these three equally, and frequently, in-school placement is less valued, understanding that prospective teachers learn these skills naturally just by being placed in a school. This way, key processes to promote reflective learning of the teaching profession, such as collaboration, interaction, monitoring, and counselling processes, are neglected.

In-school placement co-participated by university lecturers and non-university teaching tutors has a strategic character in the training of prospective teachers and sets a favorable scenario to promote the interaction between theory and practice, exercise professional skills, and experiment proposals for educational innovation monitored by experienced professionals and analyzed by educational researchers. However, the experience and the studies carried out on teaching practice show that, despite being a strategic scenario for student teachers training, the training experience can be improved, and this improvement can be implemented through the application of e-learning resources and services. For this reason, the EKT project intends to address how to improve student teachers’ training during their in-school placements through innovative methodologies and e-learning solutions that have a real impact on the following aspects that clearly need to be improved:

• Dynamics to coordinate work between university and non-university teachers who supervise and accompany students during the in-school placement period.
• Individualized follow-up of students in practice and their continued monitoring.
• An intelligent system that helps to support the self-learning process that every prospective teacher must record during his/her period of in-school teaching practice.

The project consortium was configured to respond to this challenge based on two types of partners: universities (faculties and higher education centers) that deal with initial teacher education and educational research (on educational technology and training teachers), and technology companies and institutions in the field of e-learning. This configuration has made it possible to establish a collaborative dynamic framework that allows for the project to develop appropriate solutions to the educational needs of in-school teaching practice, testing them in real contexts and therefore to produce new services and resources of e-learning that enhance higher education, and, in turn, to expand the participant companies’ competitiveness and knowledge-transfer.

In order to meet this challenge, the following objectives were defined:

• To align ICT companies’ response (in the e-learning sector) with the needs from education in Europe to articulate a better response to the challenges that education faces today. More precisely, to align advanced e-learning tools and services with active and flexible learning-teaching methodologies, which allow for a better university—school collaboration, monitoring, and counselling to result in a reflective process of in-school placement for prospective teachers.
• To establish cooperation dynamics between e-learning companies (providers of ICT and services) and researchers/experts in education (faculties of education sciences) for innovation in higher education and the ICT sector. In particular, through experimentation with their e-learning tools and services in real educational contexts, as well as through research into their uses and possibilities that allow for their improvement and adaptation to educational needs in several European contexts.
• To improve the quality of university training for student teachers during their time of professional practices (period of practical in-school experiences) through the implementation of services and ICT resources, jointly developed between e-learning companies and universities. This will be achieved through the development of flexible educational methodologies, advanced e-learning solutions that are tailored to the initial teaching processes, supporting collaboration, monitoring, and reflective learning of prospective teachers.

To achieve these objectives, the EKT project will develop new, innovative, and multidisciplinary approaches to teaching and learning, as well as university—business collaboration in the development of an e-learning technology comprehensive strategy that will be tailored to the needs of the education sector. The EKT project will also allow for the development of a collaboration partnership between universities and companies that will promote the joint creation of services based on scientific knowledge and the transfer of educational knowledge to e-learning companies, which in turn will impact on a better service to educational institutions and education professionals and the improvement of their product portfolios. The project will address an unresolved technological problem in e-learning solutions, such as the necessary interoperability of applications and services of e-learning from different sources and licenses. We intend to develop a loosely articulated system that allows for the combination of different possibilities of technologies, both open source and corporate, and e-learning services in the same environment. Today, tools and services available for e-learning offer partial and fragmented solutions, with significant communication difficulties among them. The project works on the design of a flexible, smart, versatile, and interoperable system that allows for improving the dynamics of collaboration, to regulate self-learning, making a formative evaluation and channeling the feedback to students in practice on an ongoing basis.

Finally, among its contributions, the project incorporates the integration of learning analytics aligned with the objectives of in-school practices as an advanced functionality of the EKT e-learning system. This functionality of the system will allow for collecting, organizing, processing, and providing users with a relevant set of data for the teaching−learning process that can be valuable in student teachers’ monitoring and evaluation.

2.2. Research Methodology and Status of Project Development

As it has been described, the development of the EKT project within the framework of the European call for “Knowledge Alliances” involves a high degree of technological and pedagogical innovation, and requires a rigorous research design that allows for the design of an EKT intelligent system that is based and validated to respond to the problems previously raised. Taking both issues into account, we opted for the design-based research (DBR) methodology, as it can be seen in Figure 1. This methodology is a complex approach as it relies on creative, unconventional research designs that combine multiple methods and instruments and bring out multiple data, while having great potential to impact the improvement of practice and educational research transfer.

DBR is a “systematic but flexible methodology, which aims to improve educational practices through successive approaches of analysis, design, development and implementation, based on collaboration between researchers and participants, and in real contexts leading to context-sensitive design principles and theories” [16]. Other characteristics of this research methodology include its pragmatism and its contextualized, interactive, and flexible approach.

The complexity of the research problem addressed in the EKT project, as well as the very configuration of the consortium and researchers, required a gradual and collaborative approach: the aim was to design an intelligent e-learning system that would solve the problems identified in practice and support this system with the expert and technical knowledge of the partners and in a dynamic of permanent collaboration that would align the pedagogical and technical response to the reality of the five participating European countries. This interactive dynamic of analysis, design, development, and implementation was a challenge in itself, as it combined two very different cultures (university and corporate). For this reason, a methodological architecture was designed that combined various strategies for data collection, analysis, and the development of a foundation for a quality project design that was outlined over time until the beta version of the EKT system was developed and tested in the initial teacher training centers from the five participant countries (Spain, Portugal, Ireland, England, and Austria). The sequence of research and the main decisions taken are detailed below. We are currently in the piloting phase. In this paper, we focussed on the decisions and the process carried out to design the beta-test version of the EKT intelligent e-learning system, and on the decisions taken in the definition of the necessary components and traces that will be part of the learning analytics functionalities associated to the system. This technological component of the EKT system, together with the interoperability achieved between the different tools that comprise it, constitutes a substantial advance for the development of the practices and training of student teachers from a personalized, competent, and reflective approach.

2.3. Research Phases and Instruments

The research object of this European project is currently in progress and was designed as a sequence of four phases consisting of an initial stage of analysis and exploration, followed by a phase of design and construction of the pedagogical solution (in our case, the EKT intelligent system for school practices) and a phase of evaluation and reflection on the result of its implementation. In the design-based research methodology, these three phases form an apparently linear sequence because they are the foundation for the steps of the implementation of the proposed solution, but they are interactive, and are feedback to review the analysis and design of the product to be implemented, improving it at each step, after the evaluation process, as it can be seen in Figure 2. This interactive sequence finally concludes with a double result: the final production of the implemented solution, improved from its beta to a “final” stage, and the generation of theoretical design principles that result in valuable knowledge for similar contexts, or for the better understanding of the investigated context [17].

The first phase focused on collectively defining (by academic partners and technological partners) the elements that were perceived as problematic in school practices and which also made it possible to identify the main needs for improvement and the possibilities of responding to them with the services and technological resources developed by the technological partners. In this phase, the architecture of the EKT intelligent system was defined and the lines of work of the technical team and the academic team were drawn up. During this phase, different data were collected, and systematic collaboration dynamics were established between the partners, aimed at extracting knowledge, identifying needs, and searching for consensual solutions to define the basis for the design of the EKT system. In a second phase, the educational and technological framework for development of in-school placement, the beta version of the EKT advanced e-learning system for in-school placement and a small private online course (SPOC) for academic mentors, school mentors, and student teachers were developed. In the third phase, not yet completed, the EKT system will be implemented in initial teacher training centers in five European countries (Spain, Portugal, Austria, UK, and Ireland) and all the data necessary for its evaluation will be collected. To sum up, in the fourth phase, the analysis of all the data obtained and the reflection of the process will be carried out, with the aim of improving the EKT intelligent system and developing its final version, as well as formulating design principles.

Table 1. Phases and elements involved in this DBR following the model of McKenney and Reeves (2012).

Phases	Main Elements
Phase 1 Practical problem analysis And definition of needs for improvement of in-school placements of initial teacher education (ITE).	Initial approach to the reality of school practices: Context and actors. Literature review. Foundations. Statement of the research questions and research problem. First stage of data collection: EKT Questionnaire for university and school tutors who collaborate on in-school placements. Data collection form about ITE-ISP. User requirements template. Expert panels. Expert consensus technique. Data analysis of the first data collection stage.
Phase 2 Development of the EKT advanced elearning system	Proposal of bases for the design of the intelligent EKT system (architecture, platform, and methodology). Development of the educational and technological framework for development of in-school placement. Development of the beta version of the EKT advanced e-learning system for in-school placement. Development of a Small private online course (SPOC) for Academic Mentors, School Mentors and Student Teachers.
Phase 3 Implementation of the EKT system in European initial teacher training colleges	Selection of school placement cycles, academic and school tutors and trainees (intentional sampling). Spain, Portugal, Austria, England, and Ireland. Definition of the experimentation calendar in each country. Authorization of the experimentation by the Bioethics Committees of the participating universities. Planning of the pilot experiences in each country. Implementation of the EKT proposal in the participating initial teacher training centers. Second stage of data collection. Application of the pilot evaluation tools.Pre- and post-pilot questionnaire on the use of the e-learning platform.Post pilot interviews with participants (mentors and student teachers).
Phase 4 Evaluation and reflection to produce improvements in the EKT system to formulate design principles.	Quantitative and qualitative data analysis of the second stage of data collection. Proposed improvements of the EKT advanced e-learning system based on experimentation and reflection of the participants. Development of the final version of the EKT advanced e-learning system. Formulation of design principles.

presents a summary of the design and sequence of the research, giving a complete overview of the process and the decisions taken. The first two phases of the research have now been completed and the consortium is now working on the organization of the pilots that will take place in the five European countries, and that will allow for the EKT Platform beta version to be tested. In this paper, we will refer exclusively to phase 2 of the development of the EKT platform beta version with an exhaustive description of the design process of the technological solution and of the structure and traces of learning analytics that will be incorporated in the final version of the EKT platform.

3. Results

The EKT project relies on the design and usage of an educational and technological framework that aligns the pedagogical needs for the development of high quality in-school teaching practices with the possibilities of innovation and improvement that the technologies and services of e-learning achieve.

In research phase 2, a consensus dynamic from experts was used for the construction process of the EKT system and the definition of its technological instruments, their functions, and the structure of convergence and communication among all of the tools. Two work teams were established: a pedagogical team of experts in educational technology and a technical e-learning team. In this dynamic process, it was established that there are no secrets about the identity of the issuer of the opinions, and there is no feedback directed from the outside. The criteria for choosing the experts were based on each expert having proven knowledge and great specialized experience in the design part of the specific EKT structure. Therefore, it allows for the concretion of their contributions and the establishment of a work sequence through different moments in its development. The pedagogical team, formed by experts in educational technology, is led by the University of Santiago de Compostela. The technical team, formed by experts in e-learning, is led by the Galicia Supercomputing Center. The work sequence was as follows:

• First: independent group meetings by each work team: needs analysis determination both for elements in the pedagogical system and the technological system.
• Second: joint meeting of both teams to identify the use of each system tool according to the EKT model.
• Third: individual meeting of each team to define the procedures and possibilities of using the tools to provide services to users and activities supported by the system.
• Fourth: joint meeting between both teams to design the system according to the EKT model.
• Fifth: presentation of the model to all partners from the five countries. Open debate on the activities for the internship training process and its viability in the EKT system.

This article presents the results of this phase to set the groundwork for the design base of the intelligent EKT system (architecture, platform, and methodology) that allowed for the development of the educational and technological framework for in-school placement.

The project, as we described above, was initially carried out a detailed description of the situation and profiles from the participant universities regarding their in-school teaching practices, dynamics, main actors, their roles, etc., to fully understand the situation and identify their needs. In a later phase, the University of Santiago, with the feedback of the rest of the partners, envisaged an innovative methodological proposal for the development of training in practice, as well as a set of guidelines and specifications for the definition of the e-learning system.

Therefore, there are three main results derived from phase 1 of EKT project:

-

EKT pedagogical model

-

Definition of interaction flows between main roles

-

Technological model

3.1. Result 1: The EKT Pedagogical Model

The literature analysis and the research conducted in WP2 found that the qualitative improvement of in-school placement revolves around three elements on which technologies can have the greatest impact:

• Dynamics coordinated work between academic and school mentors who supervise and accompany the students during the practice period. The two mentors (academic mentor and school mentor) intervene in the process and in a coordinated way, and throughout the whole process they will support and guide the student teacher. Other agents and groups from their educational community (practice center) and their peer group (other student teachers from their school and/or other school participating in the pilot) will also participate naturally and spontaneously.
• The individualized follow-up of students’ teachers in practice and their continued attention. The learning process has been designed with the intention of promoting and achieving an oriented and gradual involvement of the student teacher both in the school and in the teaching tasks, thus assuming responsibilities in a progressive manner: from less to more autonomy with the supervision of their mentors.
• The reflexive self-learning process that the student teacher must perform during the period of in-school teaching practice. The whole process has been designed to promote autonomous learning and reflection of the student teacher with the support of their mentors and in a learning community involving other student teachers. To this end, the EKT e-portfolio, the central tool of the process, is interconnected with all the tools of the EKT e-learning system that provide multiple functionalities for collaboration, creation, communication, training, and the realization of all the planned activities. The EKT e-portfolio will allow for collecting and guiding the reflection along the whole training itinerary of the education practice, including evidence on the activities carried out at each stage and the contributions of others (feedback from mentors, debates, and knowledge of other students’ experiences, learning materials, own productions (education practice project, materials, etc.) and tools for self-evaluation and hetero-evaluation.

The methodology and the EKT e-learning system we developed revolve around them, trying to promote and facilitate them with appropriate strategies and tools to be implemented in the pilots.

3.1.1. Key Elements of the EKT Training Model

• Coordination and communication. Coordination during practice aims to ensure that the process is developed by accompanying the student at all times (from preparation to the end of the practices) and advising them in a collaborative manner and with shared responsibility (both academic and school mentors). To achieve this, we envisaged coordination at two levels. On the one hand, at an organizational level: coordination between management teams of the universities/academies and management teams of the schools where placements will happen (practices coordinators of both institutions: academic coordinator and school coordinator) and coordination between academic and school mentors and pedagogical/training. On the other hand, coordination at the pedagogical level: coordination between academic and school mentors, coordination between the two mentors and the supervised student teacher, and coordination of each mentor (academic or school) with the group of students he/she supervises.
• Individual student follow-up. The objective of this methodology focuses on the active follow-up of the student throughout training by a team composed of the two mentors. The methodology is designed to advise and strengthen the student teacher’s individual and reflective learning. The student will use the EKT e-portfolio as a training tool to record, annotate, and incorporate training materials and materials for practice, as well as all the questions or sections that he/she decides and that are a reflection of what he/she is learning. The student will give access in his/her e-portfolio to other students in the same group so that a horizontal network of collaboration and active and reflective learning is established between them. Mentors will follow up the e-portfolio and give feedback and support to the student throughout the six stages of the EKT model. The portfolio will allow for collecting, reviewing, documenting, and interacting, and will be structured according to the methodological sequence.
• Key elements of the EKT training model. Our aim is to train teachers with a capacity for innovation and constant improvement, and this requires, among other things, to develop this capacity for reflection and analysis of their own practice. Key to this goal is the reflective process that the student carries out during placement. This process is not spontaneous: it must be guided (sequence and structure of the e-portfolio) and based on observation and on the collection of evidence by the student. They must systematically reflect on this evidence with the help of their mentors (academic and school).

3.1.2. Profiles and Roles of the Actors Involved

The results of this work are supported by the four main agents identified in the teacher’s professional practices: academic coordinator, school coordinator, academic mentor, school mentor, and student teacher. An academic coordinator is the person in charge of the management of the activities involved in student teacher’s practices before and during his/her in-school placement, such as, agreements, placements, allocation/selection of mentors, mediation with schools, etc. This function is usually carried out by a member of the management team of the initial teacher training center. A school coordinator is in charge of the management of the internship before and during its development, dealing with agreements, placements, allocation/selection of mentors, reception of the student, and coordination of students and mentors at the center, etc. These roles are generally assigned to a member of the management team at the internship center. An academic mentor is a teacher at the initial teacher training school who is in charge of the student teacher’s supervision, accompaniment, training, and assessment. This role is usually assigned to teachers with a degree or master’s degree in initial teacher education. All these three profiles are usually teachers with a university profile. Finally, a school mentor is a school teacher who supervises, supports, accompanies, trains, and assesses student teacher when in a school. Generally, this function is carried out by teachers from the placement school. In general, these are teachers from all levels of education, from kindergarten to baccalaureate. Finally, future teachers (student teachers) are students of initial teacher training certificates who, as part of their teaching degrees, carry out internships in schools. They are generally degree or master’s degree students, depending on the level of education for which they are being trained and the type of initial teacher training certificate in each country.

Participants in the EKT project focus group reported the following needs regarding management, communication, documents, analytics, and training, according to the roles described above.

3.2. Result 2: Definition of Interaction Flows Based on the EKT Pedagogical Model

The project defined some interaction flows, depending on the roles of the participant actors, which are taken into account for the guidelines and decision-making in order to create the relationships/access of the system, therefore facilitating the achievement of the pedagogical model described.

The relationship of all the practicum institutions in management through the EKT system has been organized responding to the demands of the key elements, and allows for a more organized operation and supported by the virtual space of the system between institutions and agents in the following aspects as it is also illustrated by Table 2, Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11, Table 12, Table 13, Table 14, Table 15, Table 16, Table 17, Table 18, Table 19 and Table 20:

• Access and management of all necessary information to monitor the entire training process is available to everyone without being hosted in each institution independently. This strengthens the collaborative work between participant training institutions at all times and streamlines data processing that requires a large investment of time and coordination to have it available and modify it at any time for each stage. It also allows for the consultation in real time of any of those involved to the updated information.
• It also allows for establishing the roles of each user and the uses of the system tools to work in all stages of the training process in a coordinated way.
• Furthermore, students will work with the information that contributes to their internship in the system in a continuous process, and they will avoid interacting with or even depositing duplicate documentation in each institution.
• Communications are made more flexible both at the levels of time and space, thus increasing the possibilities of the accompaniment, advice, and evaluation of the training process; one to one communication, one to all or communication between groups that are deemed necessary for specific activities.
• Generation of information, elaboration of training instruments, and creation of products is also open to all agents.
• Training is also planned and it is open to the rhythms and needs when in-school placement takes place.

Management:

Table 2. Management of schools, tutors, and student teachers.

	Manage List of Schools (Add, Edit, Remove)	View List of Schools	Create/Edit/Remove Assignments (Teacher Students and Schools)	View List of School Assignments	Create/Edit/Remove Assignments (Teacher Students and Schools)
HEI coord	Yes	Yes	Yes	Yes	All (inside HEI)
School coord	No	No	Edit contact info	Yes (inside school)	Inside school
HEI tutor	No	Yes	Edit contact info	Yes	No
School tutor	No	No	Edit contact info	Only assigned student teachers	No
Student teacher	No	No	No	Yes (only inside their HEI)	No

Table 2. Management of schools, tutors, and student teachers.

	Manage List of Schools (Add, Edit, Remove)	View List of Schools	Create/Edit/Remove Assignments (Teacher Students and Schools)	View List of School Assignments	Create/Edit/Remove Assignments (Teacher Students and Schools)
HEI coord	Yes	Yes	Yes	Yes	All (inside HEI)
School coord	No	No	Edit contact info	Yes (inside school)	Inside school
HEI tutor	No	Yes	Edit contact info	Yes	No
School tutor	No	No	Edit contact info	Only assigned student teachers	No
Student teacher	No	No	No	Yes (only inside their HEI)	No

Table 3. User management.

	Create/Edit/Remove/View Student Profile	Create/Edit/Remove/View Tutor Profiles	Create/Edit/Remove/View Coord Profile
HEI coord	All (inside HEI)	All (inside HEI)	Edit (own)
School coord	View (inside school)	All (inside school)	Edit (own)
HEI tutor	Edit (inside HEI)	Edit (own)	No
School tutor	View (inside school)	Edit (own)	No
Student Teacher	Edit (own)	No	No

Table 3. User management.

	Create/Edit/Remove/View Student Profile	Create/Edit/Remove/View Tutor Profiles	Create/Edit/Remove/View Coord Profile
HEI coord	All (inside HEI)	All (inside HEI)	Edit (own)
School coord	View (inside school)	All (inside school)	Edit (own)
HEI tutor	Edit (inside HEI)	Edit (own)	No
School tutor	View (inside school)	Edit (own)	No
Student Teacher	Edit (own)	No	No

Communication:

Table 4. Private messaging.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 4. Private messaging.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 5. Able to group with.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 5. Able to group with.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 6. Able to create new groups with.

HEI Coord	School Coord, HEI Tutor, Student Teacher
School coord	School tutor, student teacher
HEI tutor	Other HEI tutor (inside institution), school tutor, student teacher
School tutor	Other school tutor (inside institution), student teachers
Student teacher	Other student teachers (inside institution and inside school)

Table 6. Able to create new groups with.

HEI Coord	School Coord, HEI Tutor, Student Teacher
School coord	School tutor, student teacher
HEI tutor	Other HEI tutor (inside institution), school tutor, student teacher
School tutor	Other school tutor (inside institution), student teachers
Student teacher	Other student teachers (inside institution and inside school)

Table 7. Private video conferencing.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 7. Private video conferencing.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 8. Group video conferencing.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 8. Group video conferencing.

	HEI Coord	School Coord	HEI Tutor	School Tutor	Student Teacher
HEI coord	-	Yes	Yes	No	Yes
School coord	Yes	-	Yes	Yes	Yes
HEI tutor	Yes	Yes	-	Yes	Yes
School tutor	No	Yes	Yes	-	Yes
Student teacher	Yes	Yes	Yes	Yes	-

Table 9. Able to cross-videoconferencing with (create vconf-group).

HEI Coord	School Coord, HEI Tutor, Student Teacher
School coord	School tutor, student teacher
HEI tutor	Other HEI tutor (inside institution), school tutor, student teacher
School tutor	Other school tutor (inside institution), student teachers
Student teacher	Other student teachers (inside institution and inside school)

Table 9. Able to cross-videoconferencing with (create vconf-group).

HEI Coord	School Coord, HEI Tutor, Student Teacher
School coord	School tutor, student teacher
HEI tutor	Other HEI tutor (inside institution), school tutor, student teacher
School tutor	Other school tutor (inside institution), student teachers
Student teacher	Other student teachers (inside institution and inside school)

Documents:

Table 10. Create.

	Administrative Documentation	Academic Documentation	Learning Resources
HEI coord	Yes	No	No
School coord	Yes	No	No
HEI tutor	Yes	Yes	Yes
School tutor	Yes	Yes	Yes
Student teacher	Yes	No	Yes

Table 10. Create.

	Administrative Documentation	Academic Documentation	Learning Resources
HEI coord	Yes	No	No
School coord	Yes	No	No
HEI tutor	Yes	Yes	Yes
School tutor	Yes	Yes	Yes
Student teacher	Yes	No	Yes

Table 11. Share administrative documentation with.

HEI Coord	HEI Tutor, School Coord, School Tutor, Student Teacher
School coord	HEI coord, HEI tutor, school tutor, student teacher
HEI tutor	HEI coord, school coord, school tutor, student teacher
School tutor	School coord, HEI tutor, student teacher
Student teacher	HEI tutor, school tutor

Table 11. Share administrative documentation with.

HEI Coord	HEI Tutor, School Coord, School Tutor, Student Teacher
School coord	HEI coord, HEI tutor, school tutor, student teacher
HEI tutor	HEI coord, school coord, school tutor, student teacher
School tutor	School coord, HEI tutor, student teacher
Student teacher	HEI tutor, school tutor

Table 12. Share academic documentation with.

HEI Coord	HEI Tutor, School Coord, School Tutor, Student Teacher
School coord	HEI coord, HEI tutor, school tutor, student teacher
HEI tutor	HEI coord, school coord, school tutor, student teacher
School tutor	School coord, HEI tutor, student teacher
Student teacher	No

Table 12. Share academic documentation with.

HEI Coord	HEI Tutor, School Coord, School Tutor, Student Teacher
School coord	HEI coord, HEI tutor, school tutor, student teacher
HEI tutor	HEI coord, school coord, school tutor, student teacher
School tutor	School coord, HEI tutor, student teacher
Student teacher	No

Table 13. Fill administrative documentation from.

HEI coord	School coord
School coord	HEI coord
HEI tutor	HEI Coord, school coord, school tutor
School tutor	School coord, HEI tutor
Student teacher	HEI tutor, school tutor

Table 13. Fill administrative documentation from.

HEI coord	School coord
School coord	HEI coord
HEI tutor	HEI Coord, school coord, school tutor
School tutor	School coord, HEI tutor
Student teacher	HEI tutor, school tutor

Table 14. Share learning resources with.

HEI coord	-
School coord	-
HEI tutor	School tutor, student teacher
School tutor	HEI tutor, student teacher
Student teacher	School tutor, HEI tutor, student teacher

Table 14. Share learning resources with.

HEI coord	-
School coord	-
HEI tutor	School tutor, student teacher
School tutor	HEI tutor, student teacher
Student teacher	School tutor, HEI tutor, student teacher

Table 15. Learning resources.

	Able to Edit Learning Resources from	Able to View Learning Resources from
HEI coord	-	HEI tutor, student teacher
School coord	-	School tutor, student teacher
HEI tutor	School tutor, Student teacher	School tutor, student teacher
School tutor	Student teacher	Student teacher
Student teacher	Own	Own

Table 15. Learning resources.

	Able to Edit Learning Resources from	Able to View Learning Resources from
HEI coord	-	HEI tutor, student teacher
School coord	-	School tutor, student teacher
HEI tutor	School tutor, Student teacher	School tutor, student teacher
School tutor	Student teacher	Student teacher
Student teacher	Own	Own

Table 16. Follow up.

HEI coord	HEI tutor, student teacher
School coord	School tutor, student teacher
HEI tutor	Student teacher
School tutor	Student teacher
Student teacher	Own progress

Table 16. Follow up.

HEI coord	HEI tutor, student teacher
School coord	School tutor, student teacher
HEI tutor	Student teacher
School tutor	Student teacher
Student teacher	Own progress

Table 17. Involved in Student teacher evaluation.

HEI coord	No
School coord	No
HEI tutor	Yes
School tutor	Yes
Student teacher	-

Table 17. Involved in Student teacher evaluation.

HEI coord	No
School coord	No
HEI tutor	Yes
School tutor	Yes
Student teacher	-

Analytics:

Table 18. View analytics.

HEI coord	HEI level
School coord	School level
HEI tutor	Students
School tutor	Students
Student teacher	Own

Table 18. View analytics.

HEI coord	HEI level
School coord	School level
HEI tutor	Students
School tutor	Students
Student teacher	Own

Training:

Table 19. Receive training.

HEI coord	No
School coord	No
HEI tutor	No
School tutor	Yes
Student teacher	Yes

Table 19. Receive training.

HEI coord	No
School coord	No
HEI tutor	No
School tutor	Yes
Student teacher	Yes

Table 20. Prepares training for.

HEI coord	School tutor
School coord	-
HEI tutor	-
School tutor	Student teachers, school students
Student teacher	School students

Table 20. Prepares training for.

HEI coord	School tutor
School coord	-
HEI tutor	-
School tutor	Student teachers, school students
Student teacher	School students

After the detailed definition of the elements and flows for the operation with the possibilities of the tools within the system, the activities that the agents can perform within the system were established. Thus, the high education institution coordinator, has the following functions: manages paperwork to assign student teachers to school placements, defines/select/negotiate school placements to know if there are vacancies or if they are able to receive student teachers, contacts schools placements, assigns students to each school placement, coordinates practicum guides and other materials (guides, agreements, etc.), coordinates all/part university tutors, interacts with administration, interacts with school placement coordinators, interacts with HEI tutors, communication/support with student teachers at higher level (if problems with school assigned, etc.), and shares information with the HEI admin staff. The school coordinator coordinates all HEI student teachers coming to his/her school, collaborates with all school tutors and with university coordinator, manages paperwork of HEI students in school, introduces HEI student teacher to school/teachers, supervises all HEI students in school, and communicates with student teachers in school. The functions of the high education institution tutor are as follows: interacts with the group of students in a school, university coordinator, school coordinator, school tutor, and follow ups/evaluates/grades student teachers progress along with the school coordinator. The school tutor interacts with his/her tutorized students, prepares documentation for student teacher and school with collaboration of University tutor (not always), prepares training/mentors student teachers, interacts with the school coordinator, interacts with HEI tutor, and receives training from the university. Finally, the student teacher will carry out an individualized training program; create products (training materials, learning journal, contents, reports, etc.); fill in ethical docs (regarding work with underage students deontological commitment; a student can be enrolled in several practicum in the same school year); and interact with HEI coordinator, school coordinator, HEI tutor, school tutor, with the group of students in a school, with fellow student teachers in his/her practicum/degree, and with guests.

3.3. Result 3: EKT Technological Model

Based on these user roles, tasks, and rights identified in a previous phase, the EKT project proposed the following design for its technological infrastructure.

3.3.1. EKT Platform Infrastructure

The EKT e-learning system was conceived to provide support to the needs of all the in-school teaching practice stages and actors, in line with the methodology designed for the project. It is the result of the collaboration of technological partners, under the coordination of the Galicia Supercomputing Centre, who proposed implementing a multifunctional platform that integrated several open source and proprietary tools, as well as custom developments, to suit the needs of the project.

As can be seen in Figure 3, the EKT infrastructure is mainly composed of four elements under a CAS server that provides a single-sign-on authentication and entry point for all users and devices. Once logged into the system, EKT provides users with different functionalities that are provided by the following tools:

• Custom user management and support system.
• Nextcloud server, which provides cloud storage, calendar, and communication tools.
• Chamilo server, which hosts the LMS functionalities and includes an external Content Cloud server (proprietary tool from Netex company), as well as an LRS (Learning Record Store), which is a central element for the Learning Analytics functionality.

3.3.2. Learning Analytics Approach in EKT

Learning analytics is broadly referred to as the gathering of student data (interactions with others or with the system and its contents), and its analysis and visualization facilitate students to support and guide their own learning, as well as help teachers make pedagogically informed decisions and evaluations in real time. This discipline heavily relies on fields such as data mining, data visualization, machine learning, semantics, e-learning, and educational theory and practice [18]. The learning process is, in any case, a highly complex activity that encompasses a large number of factors—personal, environmental, and contextual. It is noted that it is advisable to analyze student data from different sources in order to obtain meaningful insight [19], which, undoubtedly, also results in increasing its complexity.

Among the different data models for LA, [17,18,19,20,21,22] five commonly used data model specifications for LA are highlighted, grouping them according to their main focus and possibility of providing a more complete set of data for analysis in LA. On the one hand, standards like CAM and IMS caliper focus on the learning events, and on the other hand, xAPI, activity Streams, and learning registry para data focus on learning activities. A learning event is an action triggered by the learner in a specific context, for example, when a student is interacting with a video content, several events could be triggered: play, stop, pause, rate, comment, share, etc. A learning activity, on the other hand, comprises a more complex process, involving several events, such as “groupwork”, “communication”, etc. The latter has been identified as being more relevant to the goal of providing meaningful information for the four agents of the EKT model. In this project, we adopted xAPI as the standard to use, due to our previous experience and its extensive use in building LA and its visualizations [20,21].

3.3.3. Learning Analytics Implementation Process

The development team of the project designed the following scheme (see Figure 4) to implement learning analytics so as to respond to the needs and characteristics of the educational methodology and technological framework, as well as to the calendar of activities of the project.

This plan involves seven stages from an initial installation and definition of xAPI traces to record in each of the components of the EKT platform, to the final visualization of the LA data in dashboards for the different types of users

• Stage 1. Installation of the LRS (connected to the Chamilo LMS server).
• Stage 2. Definition of the xAPI traces to record. For this stage, experts from the participant universities are consulted in order to identify what events and activities are the most relevant to identify.
• Stage 3. Detailed action list for xAPI traces will be determined. This action list determines the xAPI statements that will define the format of the specific moments in a stream of activity carried out by the different actors of the platform. This list will be validated with experts.
• Stage 4. The prior action list determined will be translated into xAPI formal statements, which will include the required information and syntax about the actor (e.g., learner), verb (e.g., watched and passed), and object (e.g., video and quiz), as well as contextual information such as timestamp and issuing authority, as well as relevant information such as results, other informational attachments, and context details (ADL initiative. (s.f.)); for example, student X has initiated a video conference with tutor Y on date Z, etc.
• Stage 5. First design of LA dashboards. A dashboard is a graphical representation of data in real time. This first approach will provide general information on the users’ activity in EKT system.
• Stage 6. User-defined dashboards. EKT consortium experts will be requested to provide feedback and suggestions on personalized dashboards for EKT objectives and approaches. All user groups present in the EKT system should be requested regarding the type of information that would be useful to achieve the project goals for enhancing learning, collaboration, and self-reflection.
• Stage 7. This final stage will integrate the different dashboards into the main EKT platform.

4. Discussion

The learning analytics approach that we developed takes into account the criticisms about the low impact, effectiveness, and context transfer in which LA has been applied, as mentioned in different research [6,7,8], where performance and control prevail over learning. The criticisms of these authors have been followed and the main actors have been taken into account by working with the information from phase 1 of the project, when a needs analysis of training agents and experts was carried out. In other words, the starting point is the pedagogical knowledge obtained based on the results of needs analysis of the educational context and of those who work in the practicum, and thus not letting technology prevail over educational interests, assuring the transfer and impact to educational reality.

The EKT (educational knowledge transfer) project aims to develop e-learning methodologies and solutions that have a real impact on clearly improvable aspects of training during the in-school placement period. For this process, the necessary components and traces have been defined that will form part of the analytical learning functionalities associated with the e-learning system that is designed as a product of the project. The project works on the design of a flexible, intelligent, versatile, and interoperable system that allows for improving collaboration dynamics, regulating self-learning, conducting a formative assessment, and channeling feedback to pre-service student teachers on an ongoing basis. This technological component of the EKT system, together with the interoperability achieved between the different tools that compose it, constitute a substantial advance for the development of practices and the training of student teachers from a personalized, competency, and reflective approach.

The process to design the intelligent system approach and the analytical learning role was proposed through a dynamic of joint work of an expert panel and tasks assigned that helped design the pedagogical and technical elements necessary for the EKT model. This design and development proposal took into account the educational implications and the learning process rather than the analytical approach [9,11]. The decision-making process in the EKT model regarding the construction of architecture technology and learning analytics was built in collaboration with both teams, which brings to the front the educational implications of the practicum training process along with their contexts. This methodology of expert working group meetings stands out for its contextualized, iterative, and flexible nature [16], in which decisions are made based on practice and understanding the educational and technical factors in its design, which will help its success in the implementation.

To obtain the results, key elements of the system were systematized and set out the interaction flows for coordination and communication, individual monitoring of the training process, and autonomous and reflective learning. Seamless access was also granted to all the tools of the system that provide either documentation, communication methods, or learning content necessary for the training.

The dynamics that were established according to the profiles allowed us to identify the participation of all and to organize the collection of student data (interactions with others or with the system and its contents) in the three key moments:

• Dynamics to coordinate the work between university and non-university professors who supervise and accompany the student teacher during the internship period at school.
• Individualized support of student teachers and their on-going monitoring.
• An intelligent system that helps to support the self-learning process that every future teacher must register during the teaching practice period at school.

It is necessary to break away from the e-learning approach for the recording of performance and assessment based only on tests results and carry out an analysis and visualization of the evidences of all the significant moments of the learning process. The evidence collected will be used to facilitate support to students and guide their own learning, suggesting learning resources and detecting undesired learning behaviors [4], as well as to help teachers make pedagogically informed decisions and evaluations in real time, and thus provide more individual attention to students [7]. It should also be noted, as the experts insist [19], to start from different sources in order to obtain a meaningful vision. In this case, it was decided that all agents have an interaction in the EKT system and then following their dynamics. In this project, all the activities where there was interaction between people, as well as with information or training activities, etc., were systematized.

A learning activity comprises a more complex process involving several events, such as “group work”, “communication”, and so on. In this EKT model, when contemplating such open interaction flows in which even colleagues enter into counseling activities and even evaluation, a more open and accessible application of learning analytics will be experienced, offering monitoring to the participants that better reflects the reality experienced by students, as proposed previously [22].

However, we found only a few studies leading in this direction [23]. Clearly, more research is needed on the implementation and evaluation of learning analytics to build on the foundation of feasibility and effectiveness [8,12].

5. Conclusions

Learning analytics is a tool with great potential to improve the learning process of students, but in order to achieve this goal, focus should be shifted from the technology itself to the learning design.

In this paper, a technological design has been proposed that starts from the analysis of the needs and characteristics of the main agents involved in the learning process. In this process, data are not only collected on the student’s progress in the technological environment, but also on their social interaction with other agents through different tools. In this sense, the approach, within the EKT system, relies on the collection of data from all the tools and systems that are part of the learning process, not only from the LMS platform, as has become customary. Proper use of the data generated is important for the EKT system and therefore, it is also considered essential to offer guidance so users can integrate this data to improve the learning process.

Finally, we consider it necessary to clarify that work with learning and data analytics has to be respectful of ethical aspects and preserve privacy and security in the use of personal data, which implies analyzing risks and anticipating situations that may lead to malpractice.