A Competency Framework for Teaching and Learning Innovation Centers for the 21st Century: Anticipating the Post-COVID-19 Age

Abstract

During the COVID-19 pandemic, most Higher Education Institutions (HEIs) across the globe moved towards “emergency online education”, experiencing a metamorphosis that advanced their capacities and competencies as never before. Teaching and Learning Centers (TLCs), the internal units that promote sustainable transformations, can play a key role in making this metamorphosis last. Existing models for TLCs have defined the competencies that they could help develop, focusing on teachers’, students’, and managers’ development, but have mislead aspects such as leadership, organizational processes, and infrastructures. This paper evaluates the PROF-XXI framework, which offers a holistic perspective on the competencies that TLCs should develop for supporting deep and sustainable transformations of HEIs. The framework was evaluated with 83 participants from four Latin American institutions and used for analyzing the transformation of their teaching and learning practices during the pandemic lockdown. The result of the analysis shows that the PROF-XXI framework was useful for identifying the teaching and learning competencies addressed by the institutions, their deficiencies, and their strategic changes. Specifically, this study shows that most institutions counted with training plans for teachers before this period, mainly in the competencies of digital technologies and pedagogical quality, but that other initiatives were created to reinforce them, including students’ support actions.

1. Introduction

“Transformation” and “metamorphosis” are in essence the same word. “Transformation” is of Latin and “metamorphosis” (μεταμόρφωσις) is of Greek origin. They both have a prefix “trans” or “meta”, which means “on the other side” or “beyond”. The second part refers to form, “forma” in Latin and “morphe” (μορφή) in Greek. Although etymologically closely related, “metamorphosis” is often used with more abrupt changes, such as the evolution in butterflies from larvae to chrysalis and adult butterflies. “The Metamorphosis” is also the chosen translation for Kafka’s book “Die Verwandlung” in languages such as English or Spanish. In this book, the main character, Gregor Samsa, wakes up one morning converted into a huge insect. He struggles with his new condition, where even simple gestures, such as opening a door, become nearly impossible for him.

In mid-March 2020, with the COVID-19 pandemic, professors all over the world must have experienced feelings such as Kafka’s character when universities across the globe moved abruptly to the “emergency online education” [1]. For better or worse, this rapid transition forced teachers to transform their teaching practices, students to adapt to new forms of learning, and Higher Education Institutions (HEIs) to advance their organization and infrastructures [2]. For a while, HEIs suffered a metamorphosis that advanced their capacities and competencies as never before.

In Kafka’s book, Gregor Samsa dies. However, some authors such as Drüke (2013) [1] believe that the title of the book refers to the transformation of Grete, Gregor’s sister, who experiences along the book a transformation to adapt to the new family circumstances. As Drüke, we believe that the metamorphosis experienced by teachers during the pandemic might serve as a catalyst for developing capabilities that last and promote a deep and sustainable transformation of Higher Education Institutions into organizations of the 21st century.

From the 1960s, institutions have invested in the creation of Teacher and Learning Centers (TLCs) as the solution for supporting this transformation. Although the concept of TLC has evolved, the spirit remains in the institutions as internal units with capabilities and positions to promote sustainable transformation of teaching and learning practices or, as Ringer defines, the hubs of educational reforms [3]. In the traditional paradigm, TLCs focus on the improvement of teaching skills and transfer of knowledge about student learning [4]. However, a new paradigm of TLCs emerged in the past years that stresses the importance of involving other stakeholders, such as students or managers, in the center activities [4], and emphasizes the idea of leadership role that they might play for having more pervasive effects [5]. Authors such as Holt et al. (2013) [5] stress the idea that TLCs “need to see their strategic leadership contribution as the designers and sustainers of open teaching and learning networks encompassing powerful forms of learning both across, and up and down the organization”.

The role of the Higher Education institutions is to develop practical knowledge to be transferred to their students to prepare skilled workforce adapted to the continuously changing market needs [6]. According to a recent study by Dondi et al. (2021) [7], today, digital skills are one of the four categories that will help students to thrive in the future of work. So, HEIs need to expose students to a variety of learning methods (online, face to face blended, theoretical, and practical) to assure job-readiness [1]. They need to assure a sustainable digital transformation, since the transformation taking place in HEIS around the globe may become the blueprint for innovation and creativity in the next decade [8]. TLCs are the units that should guide institutional transformations from the core to this end.

To support the continuous development of TLCs, prior work proposes different models defining the competencies that these centers should develop, mainly focusing on teachers’ and students’ development of digital and pedagogical practices. Although these models have shown to be very effective for defining the plans for teachers’ professional development, they are still anchored in the traditional paradigm of TLCs and mislead key aspects needed for making these centers evolve. Within these aspects, current literature highlights the need for supporting leadership and cultural change at the institution [5], the technological infrastructure for education [9], or the concept of supporting evidence-based practices to promote scholarly teaching practices [10]. There is a need for new models able to provide a more holistic perspective on the competencies of the new TLCs, more focused on providing guidelines and support for defining strategic plans for facing the challenges to come.

To advance on this challenge, this paper presents the PROF-XXI framework. This framework is one of the results of the large-scale European project PROF-XXI (http://profxxi.org/ (accessed on 21 December 2021)), which aims to build capacity for the development of TLCs adapted to the 21st century in Latin American Higher Education Institutions. The PROF-XXI framework describes the competencies that TLCs should consider for defining the strategies and actions allowing for support of teaching and learning innovation. The goal of this paper is to present and evaluate the framework with four Latin American institutions. Specifically, this paper shows how this framework can be used for analyzing the current level of competencies of an institution in terms of innovation and education from the perspective of different stakeholders. Using mixed methods, we cross-analyze quantitative and qualitative data of a workshop with 83 participants and analyzed the initiatives conducted by four TLC leaders to answer two research questions: (RQ1) How does the PROF-XXI framework helps with analyzing the competence level of the teaching and learning centers at an institution from the managers, teaching and administrative staff perspectives? (RQ2) How does the PROF-XXI framework help with identifying the competencies of the TLC developed before and during the COVID-19 pandemic lockdown? The result of this analysis aims at serving as a validation of the framework as well as a showcase for other institutions to apply it.

2. Teaching and Learning Centers: History and Models

At the end of the 1950s, international concern for teaching and learning at the higher education level was raised [11,12,13,14,15], which manifested itself in many ways: generation of taxonomies for assessment, support units for teaching and learning, vocational guidance and curriculum design, among other initiatives. However, the Anglo-Saxon universities began to configure in the 1960s [4] the Teaching and Learning Centers (TLCs) as we know them today. From the 1970s onwards, open universities have contributed to the further consolidation of the concept, by developing advanced models of teacher support and especially learner support, embedding those activities in their core organizational service structure [15,16].

In recent years, the globalization of higher education combined with the dissemination of digital technologies generated a strong political and social pressure for universities to continuously innovate their teaching and learning practices. A growing need to identify, build and develop strategic actions and contributions of support units for teachers and students has become a means for higher education institutions to guarantee quality and competitiveness [16,17,18].

Consequently, the need for frameworks to guide these practices and types of experiences has been emerging. Some of them focused on benchmarking performance [17,18], others on good practices and institutional policies [17,19]; some centered on the development and accompaniment of teachers [11,19,20,21], the systematization and maturation of learning about practice as significant referential elements for teacher development [11,20,22,23,24], or from a vision of co-creation of added value and relevance for institutions and internal collaboration [4,25]. In

Table A1. Analysis of the different models for supporting competencies of Teaching and Learning Centers (TCLs) most referred in the literature.

Framework/Model	Description
Teachers’ Focus
European Framework for the Digital Competence of Educators (DigCompEdu)	DigCompEdu was published in late 2017 by the Joint Research Centre of the European Union (JRC) (Redecker & Punie, 2017). Its main objective is to align the European educational policies with such a reference framework. DigCompEdu is a digital competence model with six differentiated competence areas: Professional engagement, Digital resources, Teaching and Learning, Assessment, Empowering learners, and Facilitating learners’ digital competence. Each area has a series of competencies that “teachers must have in order to promote effective, inclusive and innovative learning strategies, using digital tools” (Redecker y Punie, 2017, p. 4).
UNESCO ICT Competence Framework for Teachers (ICT-CFT)	This framework, developed by UNESCO, presents “a wide range of competencies that teachers need in order to integrate ICT in their professional practice” (Butcher, 2019, p. 2). It fosters practical knowledge of the advantages that ICT provides in education systems. Moreover, it suggests that teachers, apart from acquiring competencies related to ICT, must be able to use these to help their students to become collaborative, creative, innovative, committed, and decisive citizens (Rodríguez et al., 2018). This framework presents six fundamental areas or aspects of the professional teaching practice: Understanding ICT in the educational policies, Curriculum and evaluation, Pedagogy, Application of digital abilities, Organization and administration, and Professional learning.
Common Spanish Framework of Digital Competence for Teachers of the “Spanish Institute of Educational Technology and Teacher Training	The Spanish Ministry of Education, Culture and Sport launched a project in 2012 to define the Common Framework of Digital Competence for Teachers, updated four times (Instituto Nacional de Tecnologías Educativas y Formación del Profesorado, INTEF, 2017a, 2017b). It is based on the DigComp Framework of Digital Competence for Citizens (Carretero, Vuorikari, & Punie, 2017; Vuorikari, Punie, Carretero, & Van-Den-Brande, 2016). It is a generic digital competence model for educators. The competence areas (5) and competencies (21) are those of the DigComp framework.
British Framework of Digital Teaching	The British Framework of Digital Teaching was created by the Education and Teaching Foundation (ETF) in association with the JISC company (Education and Training Foundation, 2019). Its main objective is to increase the understanding of teachers in the use of digital technologies to enrich their teaching practices and improve their professional development (Pérez-Escoda et al., 2019). This framework consists of seven key areas, with three levels for each of them: exploration, adaptation, and leadership. The seven elements are Pedagogical Planning, Pedagogical Approach, Employability of the Students, Specific Teaching, Evaluation, Accessibility and Inclusion, and Self-development.
ICT Competencies and Standards for the Teaching profession of the Chilean Ministry of Education	The Education and Technology Centre of the Chilean Ministry of Education published this framework in the year 2011, as an updated version of a previous framework published in 2006 (Elliot, Gorichon, Irigoin, & Maurizi, 2011). It presents five dimensions aligned with the UNESCO Framework of ICT Competencies for Teachers (Butcher, 2019). All five dimensions work through descriptors, criteria and competencies. Moreover, each standard allows teachers to recognize how to use and integrate ICTs, identify their training needs, and define personalized training itineraries (Ríos, Gómez, & Rojas, 2018).
Framework of Implementing Collaborative Learning in the Classroom (ICLC)	The ICLC framework is based on the metacognitive framework of teacher practice by Artzt and Armour-Thomas (1998) that describes teaching in analogy to the cognitive process of solving a problem in three phases: a pre-active phase, an inter-active phase, and a post-active phase (cf. Jackson 1968). While the framework focuses on the teacher level, the student level is also presented in the framework, as the teacher’s goal is to ensure a high quality of student interaction, on which the effectiveness of collaborative learning depends (Dillenbourg et al., 1996; Kobbe et al., 2007; Webb, 1989). The ICLC framework distinguishes between five teacher competencies that span across all implementation phases of collaborative learning: the ability to plan student interaction, monitor, support, and consolidate this interaction, and finally reflect upon it.
Students’ Focus
Framework of the “International Society for Technology in Education” (ISTE) for teachers	The International Society for Technology in Education develops this competence framework focusing on the needs of the students of the 21st century (Crompton, 2017). Its main objective is to delve into the teaching practice, promote student collaboration, rethink the traditional approaches, and boost autonomous learning (Crompton, 2017; ISTE, 2018; Pérez-Escoda, García-Ruiz, & Aguaded, 2019). The general teacher profile is characterized by being active and innovative in the teaching–learning process (Gutiérrez-Castillo, Cabero, Almenara, & Estrada-Vidal, 2017). Thus, the ISTE standards for teachers are divided into seven roles or profiles that an educator must develop along his/her professional career. Framework with seven differentiated competence areas: Learners, Leaders, Citizens, Collaborators, Designers, Facilitators, and Analysts.
Managers’ Focus
ICT Competencies for Teachers’ Professional Development of Colombian Ministry of Education	The model proposed by the Colombian Ministry of Education aims to guide the professional development of teachers to improve educational innovation with ICT (Fernanda, Saavedra, Pilar, Barrios, & Zea, 2013). It is targeted at both designers of training programs and teachers interested in generating ICT-enriched environments: relevant, practical, established, collaborative and inspiring (Hernández-Suárez, 2016). This framework has five competencies that teachers must develop: Technological, Communicative, Pedagogical, Management, and Research.
Framework for the Center for Teaching Development and Innovation (Centro de Desarrollo e Innovación de La Docencia (CeDID) at the Universidad Católica de Temuco (UCT))	A framework for the evaluation of educational development programs in Chile. This framework was designed to support the diverse needs of different stakeholders: (1) faculty to make judgments about their teaching in their school and disciplinary context; (2) the learning center to evidence the impact of their educational development programs; (3) the university to inform its attainment of its planned strategic goals; and finally (4) the ministry on the effectiveness and impact of the programs that it has funded. The CeDID Evaluation Framework drew on Guskey’s five-level model, which identifies where educational development programs can demonstrate impact (Chalmers & Gardiner, 2015). These are (1) Teachers’ reaction to the development program; (2) Conceptual changes in teachers’ thinking; (3) Behavioral changes in the way teachers use the knowledge, skills and techniques learners; (4) Changes in organizational culture, practices, and support; and (5) Changes in student learning, engagement, perception, study approaches.

in Appendix A, we present several alternative models that represent these different approaches indicating the stakeholders they focus on (teachers, students, or managers).

Despite all these experiences, the gap of not having an articulated frame of reference to guide leadership and decision-making has become evident and necessary in recent times [4,18,22,26,27]. In fact, as literature shows, teaching and learning innovation can only be successfully embedded in higher education organizational culture and practices if supported by strategically driven systemic change [28]. The need to use a holistic approach for such purpose was particularly evident in the context of the pandemic as higher education institutions across the world realized they should move from randomly selected emergency remote teaching practices to more sustainable, evidence-based digital transformation processes involving strategically their entire operations, infrastructure, and staff. Recent studies analyzing the changes conducted by HE institutions during the COVID-19 pandemic also align with this idea. For example, the work by Alan and Parvin (2021) [28] proposes a policy framework for managing higher education during emergency periods based on the idea that “only a substantial policy framework will enable online technology to play a constructive role”. That is, the metamorphosis of an organization can only be complete if it involves the entire body in a process of change.

This was the background which led to the development of the PROF-XXI framework, described in detail in the following section. At the core of its design, two basic references were used, which represent the holistic and organic nature of the PROF-XXI TLC model: the European Framework for Digitally Competent Educational Organization (DigCompOrg: https://ec.europa.eu/jrc/en/digcomporg (accessed on 21 December 2021)) and the European Digital Competence Framework for Educators (DigCompEdu: https://publications.jrc.ec.europa.eu/repository/handle/JRC107466 (accessed on 21 December 2021)). In fact, the PROF-XXI framework was designed having in mind the critical interdependence that should be established between the institutional vision, strategy, and policies; infrastructure, processes, and organization; faculty, educational practices, teacher, and learner support; and, finally, quality and evaluation. These four basic pillars of higher education institution operation informed the five dimensions of the PROF-XXI TLC competencies framework (first published in [29]).

3. The PROF-XXI Framework

The change of teaching and learning practices as the metamorphosis metaphor suggests a dynamic process with different stages that could have different paces. A TLC should not only be able to address each of the different changes and paces but also be organized as a complex, multilayered, and multipurpose unit. Therefore, the PROF-XXI competencies have been structured according to five levels, each representing a TLC type of activity and stage of development. Specifically, the PROF-XXI framework proposes a set of competencies that institutions can take as a reference to develop the actions and strategies of their TLCs into these five levels and five dimensions that interrelate to define a total of 50 competencies (see Figure 1). Levels are from 1 to 5, where 1 means the lowest level of competence and 5 means the highest. See a complete definition of each level and dimensions in Appendix B.

While Level 1 “Development” defines the basic competencies that any TLC should have to start its innovation in teaching and learning, Level 5 “Public Accountability of Impacts for Continuous Improvement” refers to those competencies needed to monitor the actions carried by the TLCs to assess their impact and assure transparency. In the middle, there are Level 2 “Innovation”, Level 3 “Value Generation”, and Level 4 “New Challenges and Opportunities”. The “Innovation” level refers to those competencies able to generate and promote educational innovation at the institution. The “Value Generation” level makes explicit those competencies that will enable the institution to add value to its teaching and learning practices, generating changes that affect its culture. The “New Challenges and Opportunities” level refers to those competencies that institutions need to identify new horizons on teaching practice and quality learning scenarios, enabling the identification of indicators and metrics for evaluating these innovations.

The PROF-XXI framework also organizes the competencies into five dimensions that refer to the institutional aspects that are affected by the different competencies developed by the TLC. The “Teacher Support” (Dimension A) refers to the competencies related to the support of teachers, while the “Student Support” (Dimension B) refers to those related to students’ support. The dimension “Leadership, Culture and Transformation” (Dimension C) is one of the most innovative added by this framework. It refers to the competencies needed for leading and promoting cultural transformations at the institution through the definition of new policies and actions that affect its current processes. The dimension “Technology for Learning” (Dimension D) refers to those competencies that an institution should have to manage educational initiatives supported by technology, including the definition of technological processes and infrastructures. Finally, the dimension “Evidence-based Practice” (Dimension E) refers to the competencies needed to be able to collect data and information for understanding the effect of the transformation conducted by the institution.

The PROF-XXI framework was defined to be used in three different ways, depending on the objective of the institution: (1) as a self-assessment tool to help institutions understand their overall competence level, (2) as a reference for strategic planning definition to identify the strategic aspects to develop, or (3) as an accreditation framework to certify the innovation competence level of their TLC. This paper will focus on evaluating how the PROF-XXI framework can be used as a self-assessment method.

4. Methods

4.1. Research Objective and Design

The main aim of this paper is to evaluate the PROF-XXI framework as a self-evaluation tool to help HEIs understand the competencies of their teaching and learning centers and identify the institutional changes regarding their teaching and learning innovation policies conducted during the COVID-19 pandemic. Two research questions guided the data collection process and analysis: (RQ1) How does the PROF-XXI framework help with analyzing the competence level of the teaching and learning centers at an institution from the managers, teaching and administrative staff perspectives? (RQ2) How does the PROF-XXI framework help with identifying the competencies of the TLC developed before and during the COVID-19 pandemic lockdown?

To address these research questions, we adopted a mixed method research approach. Mixed methods are used in research that involves collecting, analyzing, and interpreting both qualitative and quantitative data from a single study to understand a phenomenon in its context. This research approach has become popular since the 1960s in disciplines such as education [30], in which a unique paradigm of research (qualitative or quantitative) is not enough to understand a complex phenomenon. For our study, we chose a fully mixed sequential method approach with a dominance of the qualitative dimension [31] to complement quantitative and qualitative data collected from a large sample of participants with qualitative information from a small sample.

We organized the research design into two phases (see Figure 2). The first phase consisted of a two-hour workshop with participants from four Latin American universities. During the workshop, the organizers introduced the PROF-XXI framework and conducted two practical activities with the participants. In the first activity, the participants were asked to answer a questionnaire to evaluate the competencies of their institution regarding the different dimensions of the PROF-XXI framework. In the second activity, they were grouped in teams of five to eight people of the same institution to compare their perceptions of the different competencies, identify activities and/or initiatives existing at their institutions and classify them into the different dimensions of the framework.

The second phase occurred 2 months after the workshop and after three researchers analyzed the data from Phase 1 to extract general conclusions of the collected data so far. The researchers prepared a document to be completed by the main leaders of the Teaching and Learning Centers of each institution. In this document they had to indicate what kind of initiatives related to education and innovation existed at their institution before the pandemic and which of them were created specifically during the pandemic and maintained nowadays.

Further details about the data gathering techniques, instruments and original data can be found in the following sections and are accessible in the Supplementary Material.

4.2. Participants and Sample

Four universities participated in this study: two universities in Guatemala (U1 and U2) and two in Colombia (U3 and U4). These universities differ in size, type of administration (public or private) and year of foundation, which represent contrasting higher education systems (see

Table A7. Original list of initiatives collected through the [Pre&Post Pandemic Lockdown Forms] translated to English. Columns of section “Periods”: Before, the initiative existed before the pandemic lockdown; Originated, the initiative was originated during the pandemic lockdown; Continues, the initiative is maintained at the institution after the pandemic lockdown. We indicated under study those initiatives that the university is still studying whether to be maintained or not.

Initiatives	Period			Competence Dimensions of the PROF-XXI Framework
	Before	Originated	Continues	A. Teacher Support	B. Students’ Support	C. Leadership, Culture and Transformation	D. Technology for Learning	E. Evidence-Based Practices
U1.1	X		X			X
U1.2	X		X	X	X	X	X	X
U1.3	X		X	X			X	X
U1.4		X	X	X			X
U1.5		X	X				X
U1.6		X	X				X
U1.7		X	X				X
U1.8		X	X	X			X
U1.9	X		X	X			X
U1.10			X			X		X
U1.11		X	X	X	X	X
U1.12			X		X	X		X
U1.13		X	X	X		X
U1.14		X	X	X	X		X	X
U2.1		X	X				X
U2.2		X	X					X
U2.3		X	X	X	X		X
U2.4	X		X				X
U2,5	X		X
U2.6	X		X			X		X
U2.7	X		X	X	X		X
U2.8	X		X	X	X		X
U2.9		X	X	X	X		X
U2.10	X		X				X
U3.1	X			X		X	X
U3.2	X			X		X	X
U3.3	X			X		X	X
U3.4	X			X		X	X	X
U3.5	X			X	X	X	X	X
U3.6		X	X	X	X	X	X	X
U4.1	X		X	X	X		X	X
U4.2	X		X	X
U4.3		X	Under study	X			X
U4.4		X	Under study	X	X		X
TOTAL	16	15	27	22	12	14	25	11
Competencies Before				12	5	7	13	6
Competencies Originated				10	6	4	12	3
Competencies Continues				15	10	9	18	9

in Appendix F for more information about the universities). They all have TLCs in charge of supporting teachers’ professional development. Eighty-three people from these four universities participated in the first phase of the evaluation.

Table 1. Participants from the different universities in the different phases.

University	1st Phase of the Analysis				2nd Phase of the Analysis
	Administrative	Manager	Teaching/Academics	Total	Teaching and Learning Center Leader
U1 (Universidad de San Carlos de Guatemala)	11	6	9	26	1
U2 (Universidad de Galileo)	8	2	-	10	1
U3 (Universidad de San Buenaventura Cali)	5	5	13	23	1
U4 (Universidad del Cauca)	1	1	22	24	1
Total	25	14	44	83	4

shows the role of the different participants in each university. Only the leader of the TLC in each institution participated in the second phase of the evaluation (four people in total).

4.3. Data Analysis

Different data were collected during the two different phases of the evaluation.

Table 2. Data gathering techniques and nature.

Code	Description	Nature of the Data Collected	Original Instrument
[Competencies Questionnaire]	Questionnaire including 50 questions in which the participants have to value from 1 to 4 each of the competencies in the PROF-XXI framework in their institution	Quantitative	[Phase1-Activity1-CompetenciesQuestionnaire-EN]: https://osf.io/zdw3e/ (accessed on 21 December 2021). [Phase1-Activity1-CompetenciesQuestionnaire-ES]: https://osf.io/ehr2t/ (accessed on 21 December 2021).
[Poster Initiatives Classification]	Collaborative digital poster created with Lucid.app for the participants to classify the different activities and initiatives conducted by their institution within the PROF-XXI framework competencies (See Appendix C). Participants had 20 min to add and discuss about the initiatives existing in their institution and associate them to a particular competence of the PROF-XXI framework.	Qualitative	[Phase1-Activity2-PosterInitiativeClassification]: https://osf.io/mfjtg/ (accessed on 21 December 2021). [ANNEX 1] For accessing the original poster used during the sessions and the main contributions.
[Pre & Post Pandemic Lockdown Forms]	For to be completed by the TLC leaders. It includes two sections: (1) a table for listing the initiatives carried out for the institution to encourage the transformation and innovation for the teaching and learning processes, indicating whether they existed before the pandemic lockdown, whether they were maintained during the this period, whether they were originated with the pandemic lockdown, whether they are currently maintained in the institution; (2) a table for indicating, for each of the initiatives in the first table, to which dimension and competencies of the PROF-XXI framework they are associated. Only those responsible of the TLC of each institution completed this form.	Qualitative	[Phase2-PosCovidForm-EN]: https://osf.io/jxhc5/ (accessed on 21 December 2021). [Phase2-PosCovidForm-ES]: https://osf.io/2trk5/ (accessed on 21 December 2021). [Phase2-U1-PosCovidForm-ES]: https://osf.io/p6rk9/ (accessed on 21 December 2021). [Phase2-U2-PosCovidForm-ES]: https://osf.io/5rac3/ (accessed on 21 December 2021). [Phase2-U3-PosCovidForm-ES]: https://osf.io/g45ty/ (accessed on 21 December 2021). [Phase2-U4-PosCovidForm-ES]: https://osf.io/s684y/ (accessed on 21 December 2021).

shows the different data sources and codes used to refer them, its nature (qualitative or quantitative), the links to the original files used for the data collection and the collected data. All the data are accessible publicly via the following link: https://osf.io/e742r/ (accessed on 21 December 2021). Three researchers participated in the different phases of the analysis for answering the different research questions addressed.

To address RQ1 about how the PROF-XXI framework helps managers, teaching and administrative staff with analyzing the competence level of the teaching and learning centers at their institution, data collected during the first phase from both the [Competencies Questionnaire] and [Poster Initiatives Classification] were analyzed. Firstly, one researcher analyzed the data from the questionnaire using Excel for calculating the average value given to each competence in the model per university (Administrative staff, Manager Staff and Teacher Staff) (

Table A3. Data analysis of [Competencies Questionnaire]. We calculated the average value given for each competence dimension for the different stakeholders and per each institution. With **, we indicate the highest values for each competence per stakeholder in each institution. The values given by the participants are between 1 (minimum) and 4 (maximum).

Institution	Role in the University	Participants	Average of A. Teachers’ Support	Average of B. Students’ Support	Average of C. Leadership, Culture and Transformation	Average of D. Technology for Learning	Average of D. Evidence-Based Practice	Average per Competence per Role
U1 (U. San Carlos de Guatemala)	Administrative	11	2.51 (SD = 0.74)	2.44 (SD = 0.82)	2.58 (STD = 1)	2.61 (SD = 0.95)	2.63 (SD = 1.14) **	2.55
	Manager	6	2.45 (SD = 0.70)	2.45 (SD = 9.57)	2.55 (STD = 0.68) **	2.65 (SD = 0.38) **	2.58 (SD = 0.62)	2.54
	Teaching/Academic	9	2.69 (SD = 0.55) **	2.47 (SD = 0.75) **	2.55 (STD = 0.82)	2.52 (SD = 0.61)	2.22 (SD = 0.84)	2.11
	Total U1	26	2.56 (SD = 0.65)	2.45 (SD = 0.72)	2.56 (STD = 0.84)	2.59 (SD = 0.72)	2.48 (SD = 0.93)	2.55
U2 (U. Galileo)	Administrative	8	3.39 (SD = 0.57) **	3.13 (SD = 0.74) **	2.95 (SD = 0.85) **	3.25 (SD = 0.72) **	2.92 (SD = 1.05) **	3.13
	Manager	2	3.00 (SD = 0.64)	2.95 (SD = 0.71)	2.82 (SD = 0.64)	3.11 (SD = 0.47)	2.75 (SD = 0.71)	2.93
	Teaching/Academic	-	-	-	-	-	-	-
	Total U2	10	3.32 (SD = 0.57)	3.09 (SD = 0.70)	2.93 (SD = 0.78)	3.22 (SD = 0.66)	2.89 (SD = 0.96)	3.09
U3 (San Buenaventura)	Administrative	5	3.99 (SD = 0.65) **	2.93 (SD = 0.53) **	2.95 (SD = 0.82) **	2.76 (SD = 0.84)	2.85 (SD = 0.76)	2.41
	Manager	5	2.67 (SD = 0.88)	2.82 (SD = 0.71)	2.93 (SD = 0.64)	2.96 (SD = 0.58) **	2.88 (SD = 0.73) **	2.85
	Teaching/Academic	13	2.69 (SD = 0.46)	2.94 (SD = 0.56)	2.94 (SD = 0.42)	2.81 (SD = 0.58)	2.72 (SD = 0.67)	2.82
	Total U3	23	2.75 (SD = 0.59)	2.91 (SD = 0.56)	2.94 (SD = 0.54)	2.83 (SD = 0.62)	2.78 (SD = 0.67)	2.84
U4 (U. Cauca)	Administrative	1	3.00 **	2.45	3.36 **	3.22 **	3.13 **	3.03
	Manager	1	2.36	2.09	2.18	2.00	2.00	2.13
	Teaching/Academic	22	2.81 (SD = 0.67)	2.72 (SD = 0.72) **	2.57 (SD = 0.68)	2.52 (SD = 0.72)	2.32 (SD = 0.87)	2.59
	Total U4	24	2.80 (SD = 0.67)	2.68 (SD = 0.70)	2.59 (SD = 0.67)	2.52 (SD = 0.71)	2.34 (SD = 0.85)	2.55
	Total general	83	2.77 (SD = 0.66) **	2.72 (SD = 0.69)	2.72 (SD = 0.72)	2.71 (SD = 0.71)	2.57 (SD = 0.85)	-

in Appendix D). Then, using these processed data, two researchers worked independently to extract a list of partial results about how institutions perceive their competence level (

Table A4. Partial results analysis of the data collected through the [Competencies Questionnaire], corresponding to the first activity of the first phase.

Partial Result Code	Description	Supporting Data Source (Table A2 and Table A3)
PR1.1	In all institutions, the competence is Competence “A. Teacher support” was valued as one of the most developed.	Competence A is evaluated with the highest values (2.77; SD = 0.66), compared with other competencies B (2.72; SD = 0.69); C (2.72; SD = 0.72); D (2.71; SD = 0.71); and E (2.57; SD = 0.85) (Table A2)
PR1.2	All institutions, evaluated Competence E “Evidence-based practices” as the least developed	Competence E is evaluated with the lowest value (2.57; SD = 0.85), compared with other competencies A (2.77; SD = 0.66); B (2.72; SD = 0.69); C (2.72; SD = 0.72); and D (2.71; SD = 0.71) (Table A2)
PR1.3	Participants from U2 and U4 evaluated the Competence “A. Teachers’ support” as the most well-developed competence in the institution, and the Competence “E. Evidence-based practices” as the least developed.	Competence A in U2 is evaluated with the highest value (3.31; SD = 0.57), while Competence E (2.89; SD = 0.96) with the lowest, compared with other competencies B (3.09; SD = 0.70); C (2.93; SD = 0.78); D (3.22; SD = 0.66) (Table A2) Competence A in U4 is evaluated with the highest value (2.80; SD = 0.67), while Competence E (2.34; SD = 0.85) with the lowest, compared with other competencies B (2.68; SD = 0.70); C (2.57; SD = 0.67); D (2.52; SD = 0.71) (Table A2)
PR1.4	In all institutions, the “Manager Staff” evaluates the competence “A. Teachers’ support” with the lowest values, together with the “Teaching Staff” from U3. However, “Teaching/Academic Staff” from U1 and U4 evaluated it as the most well-developed.	Values for Competence A for competencies and all stakeholders in the following order (Table A2): (1) Administrative: U1 (2.51; SD = 0.74) U2 (3.39; SD = 0.57); U3 (3.39; SD = 0.57); U4 (3, 00). (2) Manager: U1 (2.45; SD = 0.70) U2 (3.00; SD = 0.64); U3 (2.67; SD = 0.88); U4 (2.36). (3) Teaching/Academic: U1 (2.69; SD = 0.55) U2 (-); U3 (2.69; SD = 0.46); U4 (2.81; SD = 0.69).
PR1.5	Institution U2 has reported the highest values in terms of competence dimensions and compared with the other institutions.	Competence values in Table A2.

in Appendix D). Secondly, two researchers analyzed the list of activities indicated by the participants in the [Poster Initiatives Classification] to understand what type of activities were associated with each of the competencies in the framework.

Table A5. Partial results analysis of the data collected through the [Poster Initiatives Classification], corresponding to the second activity of the first phase.

Partial Result Code	Description	Selected Supporting Data (Translated from the Original Data)
PR2.1	To the Competence “A. Teacher support”, institutions associated initiatives for training the teachers. The types of trainings vary in frequency and format depending on the institution, including courses, workshops, seminars, and diplomas (a set of courses with several CETS credits). Most of trainings focus on learning about digital tools. Participants also associate to these competencies’ initiatives related with teaching recognition, teaching evaluation and the share of good practices.	“Training courses for teachers in new digital tools” (U1) “Creation of a support and training unit to support teachers in virtual education. Training for teachers in the use of ICT. Workshops on good practices in Moodle, Meet, zoom, classroom and other tools” (U4) “Training for teachers” (U3) “Monthly training workshops on the use of the institutional educational platform” (U2) “Sharing and supporting teachers’ successful experiences” (U4) “Learning from different experiences that led to good practices” (U4) “Evaluation on the teaching practice carried out” (U4) “Recognition of the teaching work” (U2)
PR2.2	To the Competence “B. Student support” participants associated initiatives such as online courses, video tutorials as well as academic support or on the Learning Management Systems employed by the university. Participants also recognize that, in some cases, the Competence “Student Support” is a bit poor.	“Facilitate technological tools for cooperative, collaborative and participatory work” (U1) “Only some help for internet connection” (U4) “Support to the student through a web page” (U2) “Video-lecture for the laboratory sessions” (U2)
PR2.3	To the competence “C. Leadership, Culture and Transformation” participants associated activities such as: (1) programs for developing the sense of belonging to the institution and its culture; (2) instances for self-evaluation, and instances for interacting with other institutions through research international programs. They also mentioned activities addressed to teaching/academics and administration staff related with the digital transformation of institutional processes.	“ Institutional Membership Program” (to promote the sense of belonging to the institution) (U2) “Organizational culture program” (U2) “TLC project and organizational culture focused on innovation and presentation of results and indicators” (U3) “Summa Project: Impact of the university in its context, through continuing education programs” (U4) “Culture of continuous institutional and program self-evaluation” (U2) “Each department has a person in charge of digital education” (U4). “Group work between managers and teachers for the best choice of objectives and platforms for the new modalities of virtual teaching” (U4) “Institutional training plan in competencies oriented to Technology, Communication, pedagogy, management and research” (U1) “ICT training pathway for teachers” (U2) “Workshops on good practices: In what? Teacher support leaders” (U3)
PR2.4	To the competence “D. Technology for Learning” participants associated initiatives such as training in the use of technological platforms (i.e., Moodle, Google Classroom) and tools (i.e., Google Suit) through online material, tutorials, and courses	“Support materials and tutorials for the use of digital platforms and tools” (U2) “Training in visual and audiovisual technologies, for use in virtual classes” (U4) “New tools adapted to our own institutional platform, constant innovation” (U2) “Training courses, google classroom” (U3) “Training on the use of technology in the classroom” (U1) “Implementation of technologies and educational platforms for teaching, training of students and teachers” (U3)
PR2.5	To the competence “E. Evidence-based practices”, participants associated initiatives related with the use of institutional data. The refer to initiatives for monitoring teachers and students’ performance. They also associated activities and initiatives related with the continuous curriculum improvement and benchmarking initiatives looking for other institutions practices as a reference.	“Data analysis of educational data from online courses” (U4) “Learning analytics” (U4) “Curricular design based on students’ performance” (U2) “Curricular updates at the end of the semester” (U1) “Evaluation of programs to determine innovation in teaching practice Preparation of a related semester report” (U3)

in Appendix D summarizes the partial results of this analysis, indicating some of the supporting data for each result. Finally, the quantitative results of the [Competencies Questionnaire] and the qualitative analysis of the [Poster Initiatives Classification] were triangulated to deepen our understanding of how the framework can help with analyzing the current competencies of a TLC. This process consisted of contrasting evidence obtained from the different stakeholders and from the different data sources. Three researchers participated in this process.

Table 3. Findings from RQ1. Cross-analysis of Table A2 and Table A3 for extracting findings about RQ1.

Finding Code	Description	Partial Result Supporting the Finding
F1.1	All staff in all institutions perceive that the Competence “A. Teachers’ support” is one of the most well-developed in their institution. They associated initiatives related to training the trainers (mostly for supporting the digital transition) and activities for teachers’ professional development. However, we noticed that, from all the roles analyzed (Administrative, Managers and Teaching/Academics staff), the managers were the ones giving the lowest values to this competence, while the Teaching/Academic staff in two universities (U1 and U4) evaluated them as the most well-developed.	[PR1.1] In all institutions, the Competence “A. Teachers’ support” was valued as one of the most developed (Table A3 in Appendix D). [PR1.3] Participants from U2 and U4 evaluated the Competence “A. Teachers’ support” as the most well-developed competence in the institution, and the Competence “E. Evidence-based practices” as the least developed (Table A3 in Appendix D). [PR1.4] In all institutions, the “Manager Staff” evaluates the Competence “A. Teachers’ support” with the lowest values, together with the “Teaching Staff” from U3. However, “Teaching/Academic Staff” from U1 and U4 evaluated it as the most well-developed. (Table A3 in Appendix D) [PR2.1] To the Competence “A. Teachers’ support”, institutions associated initiatives for training the teachers. The types of trainings vary in frequency and format depending on the institution, including courses, workshops, seminars, and diplomas (a set of courses with several ECTS credits). Most of trainings focus on learning about digital tools. Participants also associate with these competences related to teaching recognition, teaching evaluation and the share of good practices (Table A4 in Appendix D).
F1.2	All staff in all institutions perceive that the Competence “B. Students’ support” is one of the least developed. Participants associated initiatives such as online courses, video-tutorials as well as academic support or on the Learning Management Systems employed by the university. Participants also recognize that, in some cases, the Competence “Students’ Support” is a bit poor.	Results in Table A3 in Appendix D. [PR2.2] To the Competence “B. Students’ support” participants associated initiatives such as online courses, video-tutorials as well as academic support or on the Learning Management Systems employed by the university. Participants also recognize that, in some cases, the Competence “Students’ Support” is a bit poor (Table A4 in Appendix D).
F1.3	Despite the Competence “C. Leadership, Culture and Transformation” was not perceived as one of the most well-developed competencies; participants were able to associate some institutional activities, mainly related with the development of the “sense of belonging” to the institution, self-assessment activities, cross-institutional initiatives, and digital transformation.	Results in Table A3 in Appendix D. [PR2.3] To the Competence “C. Leadership, Culture and Transformation” participants associated activities such as (1) programs for developing the sense of belonging to the institution and its culture; (2) instances for self-evaluation, and instances for interacting with other institutions through research international programs. They also mentioned activities addressed to teaching/academics and administration staff related to the digital transformation of institutional processes (Table A5 in Appendix D).
F1.4	Participants evaluated the Competence “D. Technology for Learning” as one of the most well-developed competencies and associated activities mainly related to training initiatives in the use of institutional platforms. Most of these initiatives were addressed to the teachers/academic staff, which indicates that these initiatives are closely related with Competence “A. Teachers’ support”.	Results in Table A3 in Appendix D. [PR2.4] To the Competence “D. Technology for Learning” participants associated initiatives such as training in the use of technological platforms (i.e., Moodle, Google Classroom) and tools (i.e., Google Suit) through online material, tutorials and courses (Table A5 in Appendix D).
F1.5	The Competence “E. Evidence-based Practice” was perceived by all participants as the least developed competence in the institution. Participants associated to this competence initiatives related to the use of institutional data (Learning Analytics) for monitoring students’ and teachers’ progress and performance as well as activities related to the continuous improvement of the curriculum and benchmarking for studying initiatives in other institutions.	[PR1.2] All institutions, evaluated the Competence E “Evidence-based Practices” as the least developed. [PR2.5] To the competence “E. Evidence-based Practices”, participants associated initiatives related to the use of institutional data. The refer to initiatives for monitoring teachers and students’ performance. They also associated activities and initiatives related to the continuous curriculum improvement and benchmarking initiatives looking for other institutions practices as a reference.
F1.6	The use of the model as a self-assessment mechanism also shows that we can distinguish between those institutions with the highest and lowest competencies. In this case study, U2 was one of the institutions with the highest competencies, which is one of the institutions with more experience in the digital transformation of their teaching and learning processes.	[PR1.5] Institution U2 has reported the highest values in terms of competence dimensions and compared to the other institutions.

shows the main findings obtained from this process.

To address RQ2 about how the PROF-XXI framework helps with identifying the TLC competencies evolution before and during the pandemic lockdown, we analyzed the [Pre&Post-COVID Form] completed per the leader of the TLC at each institution. One of the implicated researchers organized in a table the different initiatives aligned with the competencies of the PROF-XXI framework and indicated whether the initiatives were created before or during the pandemic and if they have been still carried out by the institution (see Table A3 and Table A4 in Appendix D).

Table 4. Findings from RQ2. Findings for RQ2 obtained from the analysis of [Pre&Post Pandemic Lockdown Forms]. The supporting data of these findings are in Table A5 and Table A6 of Appendix E. Column period indicates the period (before, during or after the pandemic lockdown) referred in the findings.

Period	Finding Code	Description
Before the lockdown	F2.1	Before the pandemic, most of the institutions counted with long training programs for teachers (diplomas of several weeks, for example). These programs were designed for training the teachers in different areas (digital tools, pedagogical support, etc.) and are still maintained after the pandemic lockdown. However, any institution create new training programs of this type during the pandemic lockdown. Only short training programs, such as workshops for showing specific tools or training teaching methodologies, were created during this period. All these initiatives are related to Competencies A (“Teachers’ Support”) and D (“Technology for Learning”) of the PROF-XXI framework.
	F2.2	Before the pandemic lockdown, the least developed competence from the PROF-XXI framework was the Competence “B. Students’ Support” (5 initiatives out of the 16 existing initiatives before the pandemic lockdown), but the initiatives related to this competence augmented during the pandemic lockdown (8 out of the 15 originated during this period). The most well-developed were “A. Teachers’ support” (12 out of 16) and “D. Technology for Learning”.
During the lockdown	F2.3	During the pandemic lockdown, institutions invested most of their efforts in developing the Competencies “A. Teachers’ support” (10 out of 15 initiatives were related to this competence) and “D. Technology for Learning” (12 out of 15 initiatives were related to this competence); investment in Competencies “E. Evidence-based Practices” decreased (from 6 initiatives related to this competence before the lockdown, only 3 were reported associated with this competence during this period).
	F2.4	The initiatives created by the TLC before the pandemic lockdown were related with the Competencies “A. Teachers’ Support” (12 of the 16 initiatives existing in this period for all universities) and “D. Technology for Learning” (13 of the 16 in total of this period for all universities). Whereas, during the pandemic lockdown, initiatives related to “B. Students’ Support” doubled (5 out of 16 before the lockdown and 6 out of 15 originated during this period).
	F2.5	During the pandemic, all institutions created courses and materials (such as guidelines or video tutorials) for teachers and administrative staff that they facilitated through their online institutional systems. Some of the universities organized these materials in the form of online programs (i.e., U2). All universities related these initiatives to the competencies “A. Teachers’ Support”, “B. Students’ Support”, and “D. Technology for Learning”. Only U3 related the initiative created during the pandemic to all competencies of the framework.
	F2.6	During the pandemic, U1 and U2 initiated activities for supporting teachers in the use of digital tools. Examples of these activities are coaching for teachers, personalized support, etc. These institutions explicitly mentioned that they created these initiatives for promoting innovating in online assessment practices. For example, they installed Proctoring tools for facilitating online assessment. U2 related some of these initiatives to the Competence “C. Culture and Transformation”. U1 also associated some of these initiatives with the Competencies “A. Teachers’ Support” and “D. Technology for Learning” of the PROF-XXI framework.
Maintained after the lockdown	F2.7	Three out of the four universities (except U3) maintain the activities that were originated for facing the pandemic lockdown. In U4, two of these initiatives are still under study to see if they are maintained or not.
	F2.8	After the pandemic lockdown, U1, U2, U4 reported they started to use the institutional platforms (i.e., VLE, Simulators, videoconferencing, etc.) in a more systematic way. These initiatives were usually related to the Competencies “D. Technology for Learning”, and to Competencies “A. Teachers’ Support” and “B. Students’ Support” for U2.
	F2.9	After the pandemic lockdown, the number of initiatives of the TLC increased (from 16 existing before the pandemic to 27 maintained today). Although the number of initiatives associated to the different competencies increased, the universities still relate the majority of their initiatives to competencies “A. Teachers’ support” (15 out of the 27 initiatives are related to this competence) and “D. Technology for Learning” (18 out of the 27 initiatives are related to this competence), whereas Competencies “C. Leadership, Culture and Transformation” (9 out of 27) and “E. Evidence-based Practices” (9 out of 27) are still the least supported competencies.

, together with a qualitative analysis of the initiatives described, was then generated by the three researchers to extract a consensual list of findings that explains the evolution of the initiatives in the different institutions before and during the pandemic lockdown.

5. Results and Discussion

This section presents the main findings of the study, after analyzing the different data sources. Section 5.1 presents the findings related to the research question RQ1 about how the PROF-XXI framework can be used for analyzing the competence level of the teaching and learning centers at the institution from the perspective of different teaching staff. Section 5.2 presents the findings related to the research question RQ2 about how the PROF-XXI framework helps with identifying the competencies developed by the TLCs before, during and after the pandemic lockdown.

5.1. The PROF-XXI Framework as a Tool for Analyzing Institutional Teaching and Learning Competencies Development

Six findings were obtained from analyzing the data gathered in the workshop with 83 participants (administrative, teaching/academics, and manager staff) from four different institutions (See Section 3 and Section 4 in this paper). All findings suggest that the PROF-XXI framework is a good support for getting a holistic perspective of the competencies that the institution has put more effort in developing and those that are still under development. Table 4 summarizes all these findings and the analyzed data supporting them.

The first finding [F1.1] suggests that institutions invest significantly in developing “training the trainers” initiatives for developing the competencies of their teaching staff. Most of the initiatives consist of teaching programs that vary in time, frequency, and format (online, hybrid or traditional face-to-face) for training the teachers in particular competencies and promote the exchange of good practices. Therefore, the staff perceives that the competence “A. Teachers’ support” is one of the most well-developed at an institutional level. However, we observe some differences depending on their role at the university. For example, the managers are the ones giving the lowest values at this competence, whereas in two universities (U1 and U4) they rated it as the most well-developed competence [PR1.4]. This suggests that certain initiatives have more impact on some institutional staff than on others. Another possible explanation relates to the fact that this competence does not limit itself to teacher training but refers also to pedagogical and technological advising to teachers.

While the competence “A. Teachers’ support” is perceived as the most developed one, the competence “B. Students’ support” is perceived as the least developed [F1.2]. This finding suggests institutions consider that having an impact on teachers’ competencies will have a direct impact on students’ performance. Recent literature indicates that these types of approaches can have, indeed, an effect on teaching practices that influence [11] students’ perception of the learning experience [32]. However, as literature on distance education has demonstrated, learner support is paramount to cover the affective dimensions of the learning experience, along with the cognitive and systemic dimensions [14]. This explains why institutions perceive that the students’ support is still neglected and should be reinforced. After the pandemic, some literature reported that students had difficulties in following the courses in online environments [32,33]. As a response, some of the universities in this study initiated certain activities for supporting students in these new scenarios, such as video lectures or manuals on the use of their Learning Management Systems.

The competence “C. Leadership, Culture and Transformation” was also perceived in three of the four universities analyzed (U1, U2 and U4) as one of the least developed competencies [F1.3]. The participants associated to this competence initiatives related with the development of the sense of belonging, with mechanisms for self-evaluating the institution, or with activities including exchanges with other institutions as a benchmarking effort for identifying good practices. We observed, however, that the participants associated a smaller number of initiatives with this competence compared to others, suggesting that the institutions are developing some of the aspects related to this competence indirectly through other initiatives targeting other objectives. Nevertheless, this possibility indicates that leadership in their institutions is not implementing holistic, strategically driven integrated approaches as is recommended by the literature and best practices.

The participants from the different institutions evaluated the competence “D. Technology for Learning” as one of the most well-developed ones [F1.4]. Most of the institutions associated with this competence, with training programs targeting especially teachers for the development of digital skills. Although some of these courses were designed as tutorials for learning about a particular tool, many institutions reported initiatives focused on training teachers to operate in their virtual learning environments (VLEs). This suggests that institutions already have some digital strategy including the use of a VLE and other digital support, but they still need courses for promoting its usage among the teaching/academic staff. Since the engagement of teaching/academic staff with technology has been a recurrent problem in higher education [33], institutions have focused on providing support to face the resistance to change. However, prior studies suggest that teachers respond better to change when “their beliefs and practices are integrated, negotiated and reconciled with the demands of a changed context” [34]. The pandemic lockdown completely changed the context and, consequently, the teachers’ demand increased, making institutions react urgently with new initiatives that could have their effect beyond the pandemic lockdown.

Regarding the competence “E. Evidence-based practice”, the associated data suggest that participants perceived this competence as one of the least developed [F1.5]. The participants did not report many initiatives for this competence, but they associated initiatives mainly related to the use of educational data such as, for example, monitoring students’ performance and teachers’ progress. Some of them talked about Learning Analytics initiatives for promoting continuous curriculum improvement. In some cases, the participants referred to initiatives of benchmarking as a mechanism of self-evaluation and a way for looking for new practices. This finding suggests that, although this is one of the least developed competence, institutions are starting to see in educational data a good potential for supporting decision making processes [35,36]. The capacity to collect data and evidence should be complemented though with a much tighter connection between reflective teaching practice, educational research and innovation.

Finally, data supporting finding [F1.6] suggest that the PROF-XXI framework is a good support for identifying the overall competence level of teaching and learning practices in an institution and comparing it with others in a benchmarking exercise. For example, in this study, we identified institution U2 as the most well-developed and as a potential leader in the region compared to others. In fact, U2 is one of the institutions which has a larger experience and a higher level of maturity in the use of technology for digital learning and in initiatives for promoting teaching and learning innovation.

5.2. The PROF-XXI Framework as a Tool for Analyzing and Understanding the Evolution of TLCs Strategy

Eight findings were obtained from analyzing the data from the four leaders of the TLCs of the universities participating in this study. The findings obtained from this analysis show how the PROF-XXI framework can be used to understand the evolution of the TLCs competencies. Table 4 summarizes all these findings and the analyzed data supporting them.

First, findings suggest that (1) institutions should benefit from the course and initiatives created during the pandemic for updating and re-adapting their institutional plans for training the teachers to include training in those competencies required during the lockdown; (2) these courses should be complemented with learning capsules delivered in flexible formats (such as small learning capsules or online courses) to facilitate their adaptation and consumption. Findings F2.1 and F2.2 indicate that, even if institutions already put lots of effort in developing teachers’ competencies related to “A. Teachers Support” and “D. Technology for Learning” before the pandemic (especially in digital learning and quality), these competencies were not enough to face the lockdown challenges. Consequently, and as indicated by F2.3 and F2.5, institutions reinforced these two competencies during the lockdown through manuals and online courses for training teachers in particular tools. Moreover, F2.6 indicates that some institutions also implemented during this period tools such as proctoring tools for supporting new practices that they expect to maintain after this lockdown period (F2.7, F2.8 and F2.9).

These findings align with current literature, which emphasizes the importance of looking for models to adapt teachers’ training to their personal needs [11] as the only way to promote actual changes in the institutions’ culture and practices. Moreover, a recent publication shows that short online pedagogical training for university teachers has an effect on their interpretation of teaching–learning practices [33], suggesting that these types of courses could facilitate training teachers at scale in an effective manner. In addition, some authors show that the resources generated during this period can also be a mechanism to transform formal education [37]. That is, institutions have now the opportunity of benefiting from the resources developed during the pandemic lockdown to expand their training offer and effectively transform their traditional practices.

Second, findings show the importance of introducing, as part of the institutional strategy, initiatives dedicated to support students in the transition to digital education, especially in the digital competencies needed to succeed in online and hybrid practices. F2.3 indicates that initiatives related to “B. Students’ Support” were one of the most neglected competencies before the pandemic, but that institutions doubled the number of initiatives related to this competence during the lockdown (F2.4). Moreover, F2.5 shows that some of the material produced during the pandemic for supporting teachers was also associated with students’ support competence, suggesting that this material had a double purpose, to support both teachers and students.

Recent literature on the impact of the pandemic lockdown on students provides evidence that students faced various problems during this period [28,38]. Some of these problems were (1) the lack of in-home infrastructure for following online courses, especially in countries with higher socio-economical inequalities, and (2) an unfavorable study environment [32]. However, data collected after this first lockdown period show that students adapted well to the new forms of teaching and learning [32], but that institutions should still provide support for assuring the psychological well-being of students in these circumstances [32].

6. Conclusions and Implications

This paper presents the result of evaluating the PROF-XII framework as a tool for analyzing the institutional teaching and learning competencies development, as well as the evolution of the TLC strategy. The PROF-XXI framework advances the existing frameworks for supporting HEIs in their development for facing the challenges of the 21st century by providing a holistic vision of the competencies that institutions should support and develop. For the first time, a model integrates the critical interdependence between the institutional vision, strategy, and policies, on the one hand; infrastructure, processes, and organization, on the other; as well as faculty, educational practices, teacher and learner support; and, finally, quality and evaluation.

The evaluation was conducted with 83 participants from four Latin American universities (including managers, administrative and teaching/academic staff) and the four leaders of the TLCs of these institutions. The mixed-methods analysis of the collected data shows that the framework can be used as a self-assessment method for analyzing the actual development of the teaching innovation competencies at the institution through the perspective of the different stakeholders. In addition, the model was shown useful as a reference for classifying the different initiatives conducted by the TLCs and analyzing how they evolved across time, according to the emerging contextual needs. The results of this analysis indicate that the four analyzed institutions had already installed initiatives to train the teachers in digital skills and quality pedagogical practices before the pandemic, which were reinforced with short courses and learning capsules during the pandemic lockdown. These courses were created for both teachers and students since the competencies related to the support of these last ones were a bit neglected before this emergency period.

The presented work has several implications at different levels. At a theoretical level, this paper contributes with a new framework that discusses the competencies that TLCs should consider for adapting to the new societal needs and become the core of the sustainable innovative digital development at the institution. This framework proposes a holistic perspective of all the elements to be considered, which is aligned with recent policy models proposed for facing the emergency changes derived from the COVID-19 pandemic [6]. Moreover, as suggested by Alam et al. (2021) [39], HEIs are used to prioritize market-oriented outcomes, but nowadays more and more universities are focusing on promoting sustainability in their operations as well. The PROF-XXI framework offers a starting point to discuss about how operations related with teaching and learning innovation can become a daily practice in the long term.

At a practical level, the PROF-XXI framework can be used as a self-assessment tool to identify what the level of competency of a particular institution is and develop a strategy accordingly. This idea of self-effacement tool is in line with what is currently proposed by the UE platform “Digital Skills & Job Platform” [40], which proposes a test for teachers to self-evaluate their teaching digital skills. Similar solutions for evaluating what the level of competency of the institution is in terms of teaching and learning innovation could be offered using the PROF-XXI framework as a basis.

7. Limitations and Future Work

This is the first study that evaluates the use of the PROF-XXI framework as a reference tool for supporting the development of competencies in HEIs. Since this evaluation was conducted with four Latin American institutions, the main conclusions are limited to this sample. Although we included a sample of universities from two different countries and of different nature in terms of size and management, more studies with other universities could help extract further conclusions about the framework. These studies should also include students as part of the stakeholders’ analysis, since recent students show that they play a key role when conducting institutional transformations [39]. In addition, we did not analyze weather the use of this framework will produce changes in the policies in the long term. This is another interesting aspect to explore in future studies.

Future work should include analysis of other institutions. To facilitate a large-scale evaluation of the framework with other institutions, we are currently working on a web-based dashboard to visualize data of the PROF-XXI framework. This tool will facilitate the distribution of the questionnaire about the competencies employed in this study among all the educational stakeholders, including students, and compare their perception about their innovation in teaching and learning competencies. We expect that both the framework and the results of this study could help HEIs in Latin America and beyond to understand how to improve their training programs and advance on those competencies that need to be addressed for anticipating the post-COVID-19 pandemic era in a sustainable way.