Computer-Human Interaction and Collaboration: Challenges and Prospects

Abstract

Through a series of projects carried out by the Computer–Human Interaction and COllaboration (CHICO) group of the University of Castilla-La Mancha, some proposals are presented to improve the current e-Learning systems by making use of different paradigms of human-computer interaction. Synchronous and asynchronous collaborative systems, ubiquitous computing, and augmented reality can improve the current learning environments. The use of artificial intelligence mechanisms for both learner support and assessment complements these techniques. Emphasis is also placed on the use of automatic application generation techniques using models.

1. Introduction

Computers have been used for decades to teach and learn in many different branches of knowledge. Since the year 2020 and as a consequence of the COVID-19 pandemic, their use has become widespread, as they are, in many cases, the only way to attend classes, giving rise to the generalized use of asynchronous and synchronous collaborative tools that have been significantly improved since the 1990s. In an article in 2000 [1], Manuel Ortega predicted the predominance in the immediate future of collaborative systems and ubiquitous computing in eLearning from the perspective of three factors that would participate in the process: teachers, software, and hardware. In the article, the author comments on the necessary changes in the software, taking into account the following [1] (pp. 14): "On the one hand, in order to foster knowledge acquisition in a constructivist way, software should be developed aiming at real and complex projects with problem "scaffolding" and the use of the so-called "Artificial Intelligence" techniques in order to guide the students without overwhelming them. On the other hand, we must focus the solution of those problems in a collaborative way." Of the three factors mentioned above, the teacher component highlighted by Ortega has proved to be of vital importance in this crisis, and his dedication and effort in the transition from face-to-face to online classes have allowed teaching work to continue during the ongoing pandemic [2].

This article aims to review the advances of the last 20 years of research in human-computer interaction, in the areas of Collaborative Systems and ubiquitous computing in e-Learning systems, through research carried out within the Computer–Human Interaction and COllaboration [3] (CHICO) group of the University of Castilla-La Mancha in Spain, which we believe can show the evolution of these systems over the years.

The rest of the article is structured as follows: Chapter 2 presents the basic lines of Computer Supported Collaborative Learning (CSCL) systems; Chapter 3 explains the paradigm of ubiquitous computing; Chapter 4 relates to the use of engineering methods for the creation of user interfaces; Chapter 5 describes the usability of the applications; finally, some developments within the CHICO group related to the teaching of programming are presented, ending with some conclusions and proposals for the development of efficient teaching-learning systems that take into account what has been learned with the discipline of human–computer interaction.

2. Computer Supported Collaborative Learning

Collaborative learning systems are divided, in one of their most commonly used divisions, into asynchronous and synchronous. Asynchronous tools are used at different times by the participants in the interaction. A typical example of this kind of system is email. In this type of application, users utilize the tool at different times to send messages to each other, and it is not necessary to reply immediately to the received email. When the email account is opened, they can answer the received emails immediately. By not having to reply in real time, the user can think about how to respond to the asynchronous proposed action. Asynchronous tools, therefore, are used in many cases to plan solutions to complex problems in e-Learning systems, in many cases requiring artificial intelligence mechanisms. As an example, we can propose the PlanEdit tool [4] (see Figure 1), which was developed by students to solve domotics problems within the DomoSim-TPC (Domotics Simulation—Telematic Planned Collaborative) system [5]. In this case, the student solves a planning problem where they can be tutored by artificial intelligence mechanisms. In this asynchronous system, the students select actions individually in the "Individual Workspace", which are then shared in a "Collaborative Workspace" where they discuss possible solutions. When they reach a consensus, and if the solution testing mechanisms used by artificial intelligence are in accordance with what was planned, the proposal is moved to the so-called "Results Space". If the discussion in the collaborative space is too long, the system can suggest changes according to the level of scaffolding that the problem allows. The scaffolding helps the student more in the initial problems, even if they are easier, and less in the later problems presented, even if they are more difficult. In this way, help is offered according to the level of knowledge acquired by the student.

Synchronous environments [6,7] (see Figure 2) are those in which students collaborate to solve a problem at the same time, and usually from different locations. These systems usually use different forms of awareness [8] to maintain the feeling of working collaboratively, so each participant knows at all times what the other participants are doing. Figure 2 shows two students collaborating to solve a home automation problem. One of the awareness methods is the use of telepointers—red in the case of the student alumno2 and green for alumno1 (see Figure 2). In synchronous systems, the problems are solved in a limited time and therefore with decisions that are made immediately, thus requiring the solution to be planned in many cases. In this way, synchronous environments should be used after a stage of asynchronous collaboration, which allows for greater reflection on the solution of the proposed problem. These systems consider the individual contributions of the students and the collaboration between them, as well as whether the collaboration improves the results for the proposed problems, using a large number of metrics that reveal the students’ levels of development.

Students must solve home automation problems of increasing complexity, and the system uses artificial intelligence-based mechanisms to check the quality of the proposed result with an aid implemented through scaffolding. As mentioned above, the collaboration process and the quality of the solution proposed by the students are monitored on the basis of a set of qualitative and quantitative variables expressed by means of linguistic variables based on fuzzy logic. To obtain these conclusions, we use variables modeled with fuzzy sets. These sets are compound attributes or linguistic labels which define the domain of each variable. We considered variables characterized by five specific attributes corresponding to the following labels: very low, low, normal, high, and very high. In DomoSim-TPC, we incorporated the necessary functionality to draw conclusions about initiative, creativity, elaboration, conformity and disconformity.

The process-product analysis in CSCL environments carried out in the DomoSim-TPC environment is very important in any collaborative system. The quality of the proposed solution—i.e., the product—and the quality of the collaboration—i.e., whether a fruitful collaboration between the participants has taken place—must be known. Finally, it is also important to know whether the quality of the collaboration has influenced the quality of the product. To this end, a line of research in collaboration analysis was developed. It allows detailed reports of the collaborative activity to be obtained using a framework called FAPPEC (Framework for Process-Product Analysis in Collaborative Environments) [9]. This framework can be used in different tools to monitor the collaborative activity of the participants in a collaborative environment and can be defined by means of a tool with direct manipulation of the different parameters to be visualized. In Figure 3, we can see this authoring tool designed to model the intervention of a CSCL system.

Through these frameworks, precise metrics can measure the collaboration of students to solve complex problems. The metrics used come from studies of the influence of different parameters upon the collaborative process [8,9,10]. Some of these metrics for the collaborative process are:

• Number of accesses to the system that each student carried out to work in the activity.
• Number and mean of contributions made.
• Mean of the contributions’ size.
• Kind of contributions made.
• Number of replied and refined contributions.
• Depth of the discussion for each task of the problem.
• Number of times the last news was accessed.
• Number of messages sent (classified by kind: planning, coordination, or system).
• Number of instantaneous messages sent.

3. Ubiquitous Computing

Ubiquitous computing [11] is an interaction paradigm that describes different devices which collaborate through networks facilitating, in a non-intrusive way, the tasks that a user wants to perform. These devices are located wherever they are needed, hence the name “ubiquitous computing”, meaning anywhere.

An example of systems that can be encompassed within the paradigm of ubiquitous computing is the AULA (A Ubiquitous Language Appliance) system [12]. The AULA system was designed to teach foreign languages using personal digital assistants (PDAs). The students have to produce a collaborative piece of writing by dividing it in a structured way into ideas and aspects. The system can be used in or out of the classroom, and there is a synchronization mechanism for the generated papers.

AULA has a web-based version called A Web-based Language Appliance (AWLA) [13], which has been successfully used in language courses and is complemented by the AIOLE (An Interactive Online Learning Environment) system, a web-based system generator to learn foreign languages.

AWLA (A Web Language Appliance) (Figure 4) is a wiki-like system with various online language tools that allows the collaborative editing of texts in different languages with teacher correction possibilities and tools to prevent plagiarism by the learners. The whole system can be classified as a personal learning environment (PLE) with the capability to be used within the classroom, anytime and anywhere according to the paradigm of ubiquitous computing, through the use of PDAs.

4. Groupware Engineering

The ad hoc implementation of collaborative systems is an arduous task. This is why the automatic or semi-automatic creation of groupware applications is useful for the generalization of their use in classroom. The automatic creation of synchronous applications for both CSCL and Computer Supported Cooperative Work (CSCW) environments is vital to developing useful tools in a reasonable time. In this case, a model-driven engineering (MDE) method has been designed for the development of domain-independent collaborative modelling systems [10]. Figure 5 shows an application generated using the SPACE-DESIGN tool in order to allow the participants (with their picture on the right of the system in F) to generate diagrams in a collaborative way. Figure 5 also shows a shared whiteboard (A) where two participants can add elements from the application domain, in this case logic gates. Entities (B) and relationships (C) can be added and appear on the left in the environment (D, E). The telepointers (G) are used to show the area of the shared whiteboard where each participant is editing. This environment has been developed by defining models that automatically generate the final environment in Figure 5.

In order to model collaborative and interactive environments, notations and methodologies are necessary to help describe the systems that will be developed, as well as to automate the processes that generate the collaborative tools. Among the existing proposals, we highlight ConcurTaskTree (CTT) [14] and Collaborative Interactive Application Methodology (CIAM) [15].

CIAM [15] is a methodology that guides the designer to create a conceptual specification of the aspects to be considered within a groupware system in order to architect the system and design the necessary interaction in this kind of groupware system. The different steps to be carried out to obtain the final user interface can be seen in Figure 6.

CIAM (Collaborative and Interactive Application Methodology) is complemented by a notation called CIAN (Collaborative Interactive Application Notation) [15]. A summary of the CIAN is shown in Figure 7. In areas A and B, we can see the icons that represent the nodes that constitute the process model, the relationship types which indicate the different tasks, and the interdependence types. In C, we can see the icons used to represent an interaction task model in CTT notation.

The CIAN design produces a CTT diagram that in turn generates user interfaces in a semi-automatic way. To facilitate this design, a tool called CIAT (Collaborative Interactive Application Tool) [16] is used. With this tool, the whole methodological process can be followed up to the generation by means of model-based user interface design (MBUID).

The CIAN was extended to increase expressiveness in the development of e-Learning applications with the Learn-CIAN expansion and its corresponding Learn-CIAT tool [17], which can be seen in Figure 8.

5. Usability and User Experience

All the developed systems require usability studies to demonstrate their efficiency in carrying out the tasks that the user or student, as the case may be, wants to perform.

MoLEF (Mobile Learning Evaluation Framework) [18] is a proposed framework in which both the usability and the pedagogical characteristics of an e-Learning environment can be evaluated with a large number of metrics. Figure 9 shows the graphical representations of the technological and pedagogical usability parameters of a mobile application developed by the CHICO group. MoLEF has been used as a basis together with CIAM in the development of Learn-CIAN, which we discussed in the previous chapter.

Classical usability studies based on questionnaires have been completed in recent years using more objective and direct sources of information, among which we highlight the use of eye tracking techniques. The concept of eye tracking refers to a set of technologies that make it possible to record and analyze the way a person looks at a given image (or user interface), providing physiological information on aspects related to the interest, attention, and cognitive effort involved in the visual analysis of the information displayed on the screen [19].

This eye tracking technique can be used in the evaluation of educational software in the field of programming teaching and, in particular, to teach greedy algorithms [20]. This technique has also been used to evaluate awareness characteristics in groupware systems [21] used in the CIAM [22].

The use of eye tracking in the study of multimedia educational materials is a very promising field. In particular, it has been very effective in studies aimed at primary school students, in the field of mathematics teaching [23,24].

The use of eye tracking techniques has been complemented in all the case studies with classical system usability tests. In the MoLEF [18], questionnaires with Likert-type scales are intensively and extensively proposed. The set of applications from Greedex to Greedex-Tab 2.0 [20] is an example of how different user interfaces are improved by system usability tests.

In all the studies, qualitative questionnaires with a Likert scale are used to improve the usability and productivity of the proposed systems [21]. In order to measure the usability and productivity of the CIAM, usability tests were carried out among software engineers comparing these tests with other methodologies through performance of different exercises. Once the exercises were completed by the subjects, a qualitative questionnaire was provided. Each subject was required to answer some questions related to the ease of use, the perceived complexity of each model, the suitability and usefulness of the notation, and specific aspects related to some of the CIAN diagrams. Additionally, some questions about intention to use were provided. With these results, we concluded that CIAN is a notation that has proven useful to help software engineers in the modeling process of interactive and collaborative aspects in groupware applications.

The usability and productivity of different tools were also evaluated by the students, by means of questionnaires in refs. [25,26]. For the Cole Programming system [25], we used usability tests answered by the students on the different collaborative tools used. Analyzing the results, we observed how the use of forums and voting pools seems less widespread when scoring these tools.

The conclusion drawn from the use of eye tracking is that objective usability measures complement and improve classical usability analysis techniques and are of great help in the area of human–computer interaction.

6. A Case Study: Learning Environments for Programming

As an example, we will discuss learning environments for programming. Learning to program can be a difficult challenge for students in computer science. For this reason, systems based on artificial intelligence have been built in order to develop intelligent tutoring systems (ITS) such as the Cole-Programming system and COALA (COmputer Assisted Environment for Learning Algorithms) [25,26], which uses a collaborative system to teach programming on top of the integrated development environment (IDE), Eclipse [27].

Cole-Programming and COALA (Figure 10) allow the definition of a possible solution to an algorithm, and through fuzzy logic, the programming environment, implemented as an Eclipse plugin, compares the solution given by the student with the one proposed by the teacher and helps in solving the programming problems. The environment is also collaborative and allows the addition of written comments via pen-based interaction to the students’ solution.

COLLECE (COLLaborative Edition, Compilation and Execution of programs) [28] is another 100% collaborative teaching environment (Figure 11) implemented on top of the Eclipse IDE. It is complemented by a notation (ANGELA, notAtioN of road siGns to facilitatE the Learning of progrAmming) [29] that allows the visualization (Figure 12) of algorithms through a metaphor of roads and signals, which in turn allows its use to teach programming in the early stages of the students’ training. It is important to note that it can not only be utilized to teach programming, but also in peer programming environments. In addition, it encourages the creation of groups of programmers collaborating to solve complex problems.

The system described can visualize the algorithms in ANGELA notation using augmented reality Microsoft HoloLens glasses and interact with the visualized algorithm (Figure 13). This feature is highly valued by the students who can interact with the system even while debugging the code within a visual environment.

7. Conclusions

The following series of conclusions from our work can be used as a guide in the development of e-Learning systems with more capabilities than the current systems and that provide help in learning when a large part of the students’ work is carried out online.

Currently, the prevailing paradigm in online environments is the use of Learning Management Systems (LMS) or Massive Open Online Courses (MOOC), where the content is hosted in presentations or texts, and to a lesser extent with explanatory videos of certain parts of the course. Synchronous collaborative systems have also started to be used on a regular basis, especially due to the ongoing COVID-19 pandemic. A survey on educational process mining considering MOOCs, LMSs, and their relationships with collaborative systems, as well as a summary and tables relating these systems, can be found in reference [30].

However, the first recommendation of this article is the use of asynchronous collaborative systems to plan solutions to complex problems using artificial intelligence. These planning systems, and the others that will be described, need to be included within LMSs in an easy way, and for this purpose LMSs must allow the use of plugins that insert the necessary features for 21st century learning.

Scaffolding should be implemented for complex problems in the LMS. The proposed problems will be easier at the beginning, assisted by AI-based systems and will become more difficult as the students complete successive problems, whereas the help will diminish until it disappears. In this way, the students will improve their knowledge of the problems addressed.

Synchronous collaborative environments will allow the problems to be solved by several students once the problems have been planned in asynchronous collaborative systems. Again, it is vital that these systems be integrated into LMSs easily and efficiently. This is one of the results of “The Next Generation Digital Learning Environment. A Report on Research” [31]. According to Brown et al., “What is clear is that the LMS has been highly successful in enabling the administration of learning but less so in enabling learning itself”. This report argues that the Next Generation Digital Learning Environment (NGDLE) must address five domains of core functionality:

• Interoperability and integration;
• Personalisation;
• Analytics, advice, and learning assessment;
• Collaboration;
• Accessibility and universal design.

In this report, the authors explain that “The support for collaboration must be a lead design goal, not an afterthought. The current LMS is often designed on the transmission model of education—a mechanism to transmit syllabi, content, and assessments.” According to Lin et al., [32] a combination of a web-based collaborative problem-solving system and teacher guidance can be implemented to develop students’ collaborative problem-solving skills.

It is also important that we have frameworks which measure the degree of student collaboration and the quality of the answers to the problems faced, and which interrelate the two measures. In other words, we can check whether a good collaboration has produced a good solution to the proposed problem. Again, these frameworks should be integrated into LMS or MOOCs.

Creating tools that support collaborative problem solving is not feasible if they have to be implemented ad hoc. Hence, model-based user interface design (MBUID) and model-driven engineering (MDE) are needed to support the efficient development of these learning tools. Methodologies and notations specific to e-Learning such as Learn-CIAM or general ones such as CIAM-CIAN should help to define interactive and collaborative learning environments.

The user experience and usability of the proposed systems and their objective measurement are also important, which is why eye tracking systems are necessary in the development of educational tools.

Finally, we must take into account all the new interaction paradigms, such as virtual reality, augmented reality, or ubiquitous computing, so that our e-Learning tools provide real and effective learning.

In conclusion, this article presents a series of guidelines for the use of e-Learning systems that can help design future systems based on the experience of a research group in HCI and e-Learning of more than 20 years.

It is found that although the first intelligent tutor systems and collaborative systems are mature technologies, they have not been introduced conveniently in the current systems. However, their efficient use is proposed by different authors.