**1. Introduction**

Technology has been transforming education for the last decade. One of the main changes is the introduction of digital tools that support the learning and teaching practices [1]. Both software (e.g., smart tutoring systems, learning managemen<sup>t</sup> systems, educational games, simulations, or virtual/augmented reality environments) and hardware (e.g., smart whiteboards, smartphones, remote labs, robots, wearable devices, cameras and other sensors) are present in the classroom and in our daily life [2,3]. The dynamism of classrooms requires the orchestration of this complex technical ecosystem, currently performed manually by instructors. Consequentially, novel technologies and mechanisms should be considered during the deployment of flexible and dynamic smart classrooms.

These rich ecosystems collect large amounts of data about the learning process and context, opening the door to better understand and improve education. However, handling such volume of raw data also represents a complicated challenge [4]. Aware of the promises and challenges, the area

of Learning Analytics (LA) focuses on the "measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" (SoLAR definition of Learning Analytics https://www.solaresearch. org/about/what-is-learning-analytics). Within LA, over the last years, there has been a growing context on Multimodal Learning Analytics (MMLA), which is a sub-field that makes special emphasis on the usage of multimodal data sources [5]. There has been multiple and diverse MMLA applications, such as to teach how to dance salsa [6] or to assess oral presentations [7]. While transforming raw data into meaningful indicators is already daring [4], in this manuscript, we are mostly concerned with the issue of orchestrating the different data sources and applications. A recent literature review on MMLA architectures reveals that, due to the complexity of orchestrating the different elements of the technical ecosystem, most of the proposals offer ad-hoc solutions [8]. Apart from limiting the chances of reusability in different educational contexts, the effort to develop, deploy, maintain and enable interoperability among all those ad-hoc solutions does not scale up when the number of solutions increases [9]. Therefore, the current ad-hoc setup represents an important challenge to systematically apply MMLA in smart classrooms [10].

Thus, a real futuristic scenario with smart classrooms, where consecutive lessons take place (with 15–30 min breaks), would require a seamless and scalable reconfiguration of the sensors, devices and virtual learning environments within the classroom not only to deliver the lesson but also to profit from highly different MMLA solutions [10]. To address these challenges, we propose to evolve from traditional management, predefined by the instructor in a manual fashion, towards an automated approach able to reconfigure the classroom devices without human intervention and in a flexible and on-demand way. The number of sensors and actuators making up smart classrooms, as well as the possibility of managing them in a dynamic way make the scalability of the proposed approach a critical aspect to take into account. This can be possible by deploying a Mobile Edge Computing (MEC) architecture that combines Network Function Virtualization (NFV) technique [11] and Software-Defined Networking (SDN) paradigm [12]. NFV will allow for separating the software logic from the hardware of the classroom devices. It improves the flexibility and dynamism of device managemen<sup>t</sup> processes by enabling the deployment, dismantling and reconfiguration of the technical ecosystem according to the current classroom needs. The SDN paradigm will help smart classrooms with automatic and dynamic managemen<sup>t</sup> of network communications, enabling the Quality-of-Service (QoS) and interoperability of smart classroom devices and applications at the edge.

The objective of this paper is to present an MEC-enabled architecture that integrates SDN/NFV to deploy, configure and control the lifecycle of MMLA applications and devices making up a smart classroom as well as its network communications at any time and on-demand. More specifically, the objectives of this paper are as follows:


The remainder of this paper is structured according to the next schema. Section 2 reviews and analyzes the state of the art of smart learning and classrooms, MMLA, remote smart classrooms, as well as the usage of SDN and NFV in different scenarios. Section 3 shows a case study explaining three different scenarios and their concerns. Section 4 describes the proposed architecture and how it can address the concerns of the aforementioned scenarios. Section 5 presents some experimental results that demonstrate the usefulness and performance of our solution. Section 6 discusses the main benefits

of our solution compared to the existing ones. Finally, conclusions and future work are drawn in Section 7.

#### **2. Related Work**

#### *2.1. Smart Learning Environments and Classrooms*

In the last few decades, multiple terms have been coined with the "smart" label, often referring to devices (such as phones or watches) or spaces (e.g., classrooms, schools, campus, or cities) that through the utilization of the appropriate technologies and Internet of Things (IoT) services collect data from the users and the context to better adapt to the needs of the stakeholders involved. Aligned with this general idea, Smart Learning Environments (SLEs) are technology-enhanced learning environments able to offer instant and adaptive support to learners based on the analyses of their individual needs and based on the contexts in which they are situated [13]. Thus, when we think of a *smart classroom*, we should not reduce it to the mere idea of a traditional classroom heavily equipped with virtual learning environments and mobile, wearable or IoT devices.

While many aspects should be taken into consideration in a smart classroom such as the architectural design and its ergonomy, or the pedagogical methodology [14], in this paper, we focus on the infrastructure required to enable the "smart" features, i.e.,: (1) to seamlessly reconfigure such a complex technological infrastructure for guaranteeing the dynamicity and QoS of smart classrooms; and (2) to collect data from users and context to feed data for the intelligent adaptation to the learning needs at enactment time.

#### *2.2. Architectures for Smart Learning Environments and Classrooms*

As a recent literature review on smart campus technologies shows [15], paradigms and technologies such as the IoT, virtualization, wireless network, or mobile terminals are essential parts to be considered. There have been several attempts to orchestrate this intricate technical ecosystem. At the beginning, many of them were ad-hoc architectures suitable for specific technologies (e.g., interactive boards [16]), or focused on concrete problems (e.g., communication issues [17,18]) or features (e.g., remote software control [19]). Lately, authors have started broadening the scope and flexibility of their proposals. For example, GLUEPS-AR [20,21] combines the lessons learnt from distributed learning environments and the ideas coming from the MMLA domain. In [21], Serrano et al. designed an architecture which gathers student actions and their contextual data during across-spaces learning tasks to feed the adaption features. Another example is the architecture proposed by Huang et al. [22], which not only conducts the collection, integration and analyses of contextual data, but also enables the remote control of IoT devices and enhances the usability of the smart classroom with additional services such as voice recognition and user control interfaces. Previous colleagues also introduced LEARNSense framework [23], which aims to provide learning analytics using wearable devices. However, they did not deal with scalability and deployment issues either.

These architectures often focus on supporting data processing activities of the Data Value Chain (DVC) [24] (namely, collection and annotation, preparation, organization, integration, analysis, visualization, and decision-making). Each of these data processing activities poses a number of challenges linked to the problems associated with the data collection and analysis of multimodal data sources [8], which are common in smart classrooms. However, none of these proposals details how to (re)configure the smart classroom technical ecosystem to seamlessly switch from one LA application to another. Thus, in this paper, we try not only to enable the DVC in a smart classroom but also to reconfigure the technical ecosystem to cope with the requirements of different lessons happening in a row.

#### *2.3. Remote Classrooms and Labs*

Related to the technical orchestration challenges of smart classrooms, the virtual and remote lab field has a long trajectory coordinating IoT services and devices. Remote smart classrooms consider virtualization techniques and virtual machines (VM) to optimize the managemen<sup>t</sup> of their software and hardware resources flexibly. Some remote laboratories consider virtual labs as an essential tool to improve the learning experience by supporting experimentation about unobserved phenomena [25]. In [26], the WebLab-Deusto project [27] used VMs to provide their students with remote smart laboratories that do not consider WebLab-specific code. Students had access to VMs for a given time and, once finished, a snapshot was made before restoring and preparing the VMs for new students. In [28], the authors proposed a solution that considered virtualization techniques to adapt the resources of remote laboratories at anytime and on-demand. Several experiments demonstrated how the usage of computing resources was optimized to guarantee the smart labs quality of service. In [29], the authors presented a mechanism to automatically generate, deploy and publish digitized labs in a framework of Massively Scalable Online Laboratories (MSOL). The authors demonstrated the suitability of the proposed mechanism by developing a communication protocol managing the lab equipment remotely, together with a web platform enabling the managemen<sup>t</sup> of files and publishing digitized labs as web applications. Finally, the Smart Device Specification [30,31] provided remote labs with interesting capabilities. This specification focused on removing dependencies between clients and servers while enabling the description of remote lab experiments, and the selection of particular remote lab configurations [32]. However, configurations were not flexible enough because these must be established in advance by the lab administrator.

#### *2.4. SDN and NFV Applied to Different Scenarios*

The combination of SDN/NFV enables flexible, dynamic and on-demand managemen<sup>t</sup> of networking and infrastructure resources. Moreover, it facilitates disruptive and heterogeneous scenarios such as the next generation of mobile networks (5G) [33], healthcare environments [34], or IoT [35].

Regarding 5G mobile networks, the authors of [36] analyzed the impact of SDN/NFV in the new vision of current and future network architectures. The authors highlighted how the combination of SDN/NFV reduces costs while improving the network flexibility and scalability of the infrastructure. The authors of [37] proposed a 5G architecture using NFV to support the implementation of tactile internet. A utility optimization algorithm which enables human perception-based tactile internet was developed to optimize the utility of 5G NFV-based components in this new scenario. In [33], the authors proposed an architecture which integrates SDN/NFV to manage and orchestrate services in charge of monitoring and controlling the network plane of a 5G network infrastructure in real-time and on-demand. Another solution was presented in [38], where authors studied the network flows migration of 5G networks and pointed out the inverse relationship between network load balancing and reconfiguration costs. Several experiments demonstrated the previous trade-off and the usefulness of the proposed solution. Regarding healthcare scenarios, the authors of [34] proposed an SDN/NFV architecture providing flexible and cost-efficient deployment and control of healthcare applications and services. In addition, the authors of [39] proposed an SDN/NFV framework to control the life-cycle and behaviour of physical and virtual medical devices belonging to clinical environments. This work also presented the novel concept of virtual medical device, an NFV-aware system providing dynamism in clinical environments. In the IoT context, the authors of [35] introduced an SDN/NFV architecture providing IoT devices with ultra-low communication latency. Another work was proposed in [40], where authors designed an architecture to ensure key security and privacy aspects of cyber-physical systems and IoT environments. SDN and NFV were considered to allow IoT devices and environments to make security decisions and take dynamic reactions. It is important to mention that learning scenarios such the proposed in this work can be improved by considering the SDN/NFV capabilities presented in the previous works.

In conclusion, this section has reviewed some of the most relevant solutions of heterogeneous smart learning environments and remote classrooms, highlighting the importance of seamless reconfiguration of smart classroom devices. The lack of solutions able to deploy, dismantle and reconfigure the software of classroom devices has also been demonstrated to ensure the seamless reconfiguration of devices in real-time and on-demand. Finally, we have shown the potential of SDN and NFV in other scenarios to achieve flexible and dynamic managemen<sup>t</sup> of computational, storage and networking resources.

#### **3. Description of Simulated Case Study**

The related work review concluded that one of the main challenges in the area of smart classrooms and MMLA context is an architectural one. In our attempt to understand in depth this research issue, this section presents a simulated case study inspired by authentic uses cases extracted from the literature. The main goal is to ascertain what the specific issues are that our proposed architecture must address (see in next Section 4), in order to support a seamless reconfiguration of a smart classroom where different learning activities would happen in a row. With the objective of building this case study, we reviewed literature on MMLA applications that have been implemented during the last few years. From these cases, we select three that were aligned with innovative learning trends and have different objectives, devices, analytics and sensors in order to demonstrate how the architecture self-organizes from one scenario to the following one. We also order these three cases by increasing complexity, the first one focuses on individual students, the second one focuses on groups of students collaborating, and the third one focuses on students collaborating in projects but also on what the instructor is doing. Next, we describe in depth each one of the scenarios.

#### *3.1. Intelligent Tutoring System in the Classroom*

One of the main trends in education over the last decade has been the development of interactive environments that can be slowly introduced as part of the classroom or homework activities. Two of the most relevant tools for this purpose are Intelligent Tutoring Systems and Educational Games [41]. Most of the literature meta-reviews that have measured the effectiveness of such tools in the classroom [41,42] have reported positive effects. However, these studies also agree on the struggle that instructors face to effectively integrate these tools in their teaching and curriculum. One of the reasons is not being able to know what students are doing in these virtual environments to orchestrate the classroom activities and to intervene if necessary. Hence, the need for the development of real-time dashboards that can provide this information to instructors [43].

The first scenario is grounded in this technological and pedagogical issue, and is strongly inspired in the previous work of Holstein et al. [44,45]. In this work, they have co-designed a dashboard and augmented wearable instruments to visualize real-time analytics and visualizations of what each student is doing in the intelligent tutoring system. Next, we detail the specific details:


#### *3.2. Tabletop Task Collaboration*

UNESCO has noted that the future of education should be focused on promoting transverse skills, such as collaboration [46]. The trend has shifted from individual efforts to group work, making the development of collaboration skills mandatory with an increasing trend of implementing collaborative learning activities with high frequency [47]. Therefore, it is not a surprise that numerous researchers have started to analyze collaborative learning from different perspectives. However, one of the challenges has been to scale up the analysis of these collaboration studies when there are many groups to assess or to provide feedback in real-time. Hence, the area of MMLA has been studying ways to automatically provide empirical evidence that can help to support co-located collaboration through analytics [48]. In these studies, researchers capture multimodal data from the collaboration, some examples of data sources include video, audio, physiological signals using wearables or interaction data with computers or shared devices [49,50].

This second scenario is grounded in this context where we present an application that generates colocated collaboration analytics while students are interacting on a multi-touch tabletop doing a collaborative task, which is based on previous work from Maldonado et al. [51]. The details of this scenario are depicted next:

	- **–** *Group multi table-top*: Table-top learning environments are big tactile screens that allow the collaboration of multiple users at the same time.
	- **–** *Group overhead depth sensor*: A Kinect sensor is used to track the position of each user automatically detecting which student did each touch.
	- **–** *Group microphone array*: It is located above the tabletop and captures the voice of all the group members, distinguishing the person which is speaking.

**–** *Instructor device*: The instructor consume the analytics via a dashboard by connecting from its device (tablet or laptop) to the visualizer provided by the architecture.

#### *3.3. Programming Project-Based Learning and Instructor Indoor Positioning*

Project-based learning has become one of the main forms of instructions across contexts and the different phases of schooling as it resembles better real-world practices and leads to deeper learning [54]. This method of instruction is very common in programming courses, where students often have to develop a collaborative group programming project to pass the course (e.g., [55]). One of the challenges of these collaborative projects is to assess the role and effort of each member of the group in order to guarantee similar workload distribution, hence avoiding free riding [56]. These project-based learning courses often have entire sessions devoted to in-class work on the projects. During these sessions, the teacher moves from group to group solving doubts, which presents a new challenge regarding how to equitably distribute their time across groups [57]. In this context, we can collect diverse sources of data from the collaborative programming environments, audio from group conversations, instructors' position and physiological signals from the students.

This third scenario combines inspiration from the following previous studies: the work of Spikol et al., and Blikstein to apply MMLA to analyze collaborative project-based learning and open-ended programming tasks [58,59], the ideas of Ahonen et al., to analyze biosignals during these programming tasks [60] and finally the proposal of Martínez-Maldonado et al. to estimate the amount of time spent by the instructions in each group [57]. Therefore, in this scenario, the application combines an analysis of the collaborative programming actions and conversation of each group, the physiological signals levels of each student and position of the instructor. More details about this scenario are depicted next:

	- **–** *Individual students' devices*: Students interact with the collaborative programmings environment by connecting to it through a web application.
	- **–** *Individual Empatica E4 wristband*: Each student wears an E4 empatica wristband that captures the heart rate, a three-axis activity through an accelerometer, and the electrodermal activity of their skin.
	- **–** *Group microphone array*: It is located above each one of the groups' tables, distinguishing the person which is speaking.

#### *3.4. Requirements of the Previous Scenarios*

The case study with the three consecutive scenarios represents an example of how smart classrooms and MMLA solutions could look in the future. To reach our goal of supporting the seamless reconfiguration and data collection required to enable the smart adaptation, we have identified four main requirements emerging from our simulated case study:

**Requirement 1—Within-scenario flexibility for instructor-configured data collection, analytics, visualizations, and recommendations**: Aligned with the challenges reported in the literature [8], the MMLA solutions implemented in the aforementioned scenarios are ad-hoc solutions that enable the data gathering and analysis to later feed the visualizations and recommendations for instructors and students. The three use cases that we described have different learning environments, devices, data sources or analytics pipelines that have been configured to match the necessities of each use case. Therefore, to be able to scale up the number of MMLA solutions used in a single classroom and scenario, it is necessary to provide a scalable architecture compatible with the different MMLA applications [9,10] by abstracting these functionalities in scalable and interoperable modules that can be automatically re-configured for each MMLA application.

**Requirement 2—Between-scenario flexibility for automatic deployment of the MMLA solutions**: The kind of equipment, devices, setup and sensors necessary to perform these applications makes smart classrooms expensive to have. Therefore, we would expect that, in the future, these classrooms are fully booked, perhaps having a short time of 15–30 min in-between sessions. In our case study, we presented three consecutive use cases to illustrate this issue, but this might be a conservative estimate. The current setup makes it very challenging to seamlessly and automatically re-configure the technical ecosystem and to also enable the data collection and analysis in short periods of time. In our case study without a proper architecture, each teacher would be in charge to deal with the technological complexity of the MMLA application in each class, which in reality is not a feasible approach. This raises the necessity to have a seamless transitions between the scenarios of our simulated case study.

**Requirement 3—Seamless privacy and authentication configurations**: The privacy of users, and of students in this case scenario, has been one of the topics on the spotlight during the last years [62]. The regulations have agreed that we need to provide control to the users so that they can specify how their data can be used. Therefore, even though these MMLA solutions seek to help students in their learning process, students and instructors should still have the right to opt-in or -out so that their data are not collected and/or used. In the case scenario, each application would need to manage this privacy and authentication issues separately, which is sub-optimal. Therefore, we need to provide a centralized system where students can configure their privacy and authentication options to apply across all the smart classroom applications, and we also need to easily identify students across applications and devices so that we can properly process their data.

**Requirement 4—Easy communication with external data sources**: Thanks to the institutional data and the ICT adoption in our daily routines, there can be numerous data sources (both formal and informal) that can hold valuable information to understand students' context and knowledge. Some examples might include the classical LMSs in formal learning institutions, other online courses, academic records or background information. In the case study, each application would

have developed their own interface to interact with these external data sources. Thus, instead of implementing ad-hoc solutions to benefit from those external data sources, there is a need for generating services and APIs that can be used across applications.
