E-Learning Enhancement through Multidisciplinary Teams in Higher Education: Students, Teachers, and Librarians

Abstract

The societal disturbance created by the rapid outbreak of the COVID-19 pandemic has shaken the entire globe, profoundly affecting all levels of education. The challenge presented by COVID-19 is broad, rapidly evolving, and complex; it threatens everyone’s well-being, the global economy, the environment, and all societal and cultural standards and our daily activities. Throughout the Coronavirus outbreak and any future lockdowns, it is crucial that the needs of students be ultimately and regularly met and that they are supported effectively. We intend to address skill shortages and mismatches, particularly regarding the readiness to teach in an online environment that encourages flexible and innovative learning. The main contribution of this paper is addressing this subject with an integrated vision of three different players in higher education: students, teachers and librarians. Using the Technology Adoption Model (TAM) and the Unified Theory of Acceptance and Use of Technology (UTAUT), a conceptual model was developed to explain both the behavior and intentions of users when using e-learning systems. Among Portuguese students, 91% of e-learning satisfaction can be attributed to perceived usefulness, actual use, and personal considerations. For educators, satisfaction appears to be mostly dependent on perceived usefulness and usability, while librarians’ satisfaction is negatively dependent on technological factors. Students’ actual use of technology is 89% dependent on organizational and technological variables. However, the actual use by teachers appears to be primarily dependent on personal and technological factors. Similarly, 91% of the variability of the use of e-learning tools by librarians can be explained by organizational, personal and technological factors, with the personal factors having a negative impact on the actual use.

1. Introduction

The crisis caused by the COVID-19 pandemic has compelled European universities to relocate their instructional operations online. Although most Higher Education Institutions (HEI) implemented e-learning platforms years ago, the teaching staff is now encountering difficulties in using those platforms and generating and altering course content, which is required to adjust to a rapidly growing and complex situation.

E-learning has evolved dramatically throughout the years. According to Alqahtani and Rajkhan [1] (p. 1), “Prior to the COVID-19 pandemic, E-learning was growing approximately 15.4% yearly in educational institutions worldwide without uncertainties or pressure on those institutions or students”. Nonetheless, e-learning reached a significant apex with the emergence of the pandemic. Due to the substantial risk of contamination, many nations have taken steps to minimize face-to-face interactions in educational contexts, shifting from a face-to-face model to a comprehensive e-learning technique [2].

It is important to consider the responses and problems in e-learning processes seen in HEIs in different countries during the COVID-19 pandemic and to use this experience to improve future practice. The Digitools project fits this assumption. Digitools is a European project aiming at offering improved digital tools and methodologies to aid HEI in providing quality online education. This project supports and cultivates innovative pedagogies and methods for teaching, learning, and assessment, encouraging educators and students to use digital technologies in creative, collaborative, and efficient ways to help them quickly adapt to this rapidly evolving and complex situation that results from this global pandemic. HEI information services, namely academic libraries, have the potential to increase the integration of digital technologies in teaching processes, through a more active role in the development of students’ and teachers’ digital skills and the availability of online educational resources.

With this focus academic libraries will help to consolidate success and ensure scalability and sustainability to assist educational institutions in the EU in reevaluating their organizational policies to increase their innovation capacity and maximize the potential of digital technologies and content.

This initiative aims to assist HEIs in acquiring the skills and abilities required to design and deliver high-quality online courses, including blended instruction. The HEI libraries play a significant part in this project, as they are encouraged to modify their services and offer digital learning materials and information and digital skills training to teachers and students. As a result, HEIs can offer high-quality online training courses emphasizing subject-specific instruction through their teaching staff and libraries.

Looking at Portugal in particular, e-learning has been experiencing a growing process. It started long before COVID-19 but became widespread due to the emergency lockdown. In fact, Bastos [3] studied the perceptions, barriers, and opportunities of e-learning from Portuguese students during this context. However, despite the fact that e-learning is quite present in Portugal, especially in higher education, there are no references to the involvement of Portuguese academic libraries in this process. On the contrary, in some countries, librarians play an important role in e-learning, by providing electronic information resources, online reference services, creating mobile tailored-made content to be accessed on mobile devices, and providing online courses to promote skills on the use of information databases [4,5].

While analyzing the e-learning state of the art in Portugal, with an integrated view of students, teachers and librarians, we intend to determine the readiness of the institutions to undertake e-learning projects, to prepare the organizations for these projects, and to improve their e-learning strategies and procedures.

2. Theoretical Background

To overcome the various obstacles, including the pandemic, Internet and software-based resources have increased their popularity. The widespread use of smart phones and tablets, multimedia platforms, software programming and other technologies presents new possibilities for teaching. E-learning provides unparalleled accessibility, unrestricted by location, faculty availability, time constraints or cost to the learner. In addition to the educational advantages of distance learning, some authors highlight macro advantages such as financial and environmental benefits, by reducing the travel of individuals to educational institutions [6].

According to Gautam [7] and Mukhtar [8], e-learning can be easily managed, and the learner can easily contact the teachers and access teaching materials.

Radha et al. [9] show that e-learning has become popular among students in all educational institutions during the pandemic lockdown period. The students’ positive attitude towards e-learning is mainly due to the feeling of improvement that comes from self-study skills, their satisfaction regarding online mock tests, and also because they acknowledge the usefulness of e-learning during the quarantine period. E-learning also promotes valuable learning outcomes such as higher-order thinking abilities and more autonomous learning time management, and favors psychological motivation, peer collaboration, cognitive problem solving, interaction with instructors, and community support [10]. Another e-learning advantage seems to be the easiness and speed with which it is adopted. For example, in a Romanian HEI, Edelhauser and Lupu-Dima reported that “students have adapted quickly to virtual education, and between March and May 2020, 87% of them participated in online courses” [11] (p. 27).

However, e-learning also poses new concerns related to the security and the reliability of technologies, in addition to other difficulties related to the misuse of technology. According to Somayeh et al. [6], the main obstacle regarding e-learning usage is the absence of crucial personal interactions, not only between students and teachers but also among fellow students. Furthermore, it seems that students tend to prefer a conventional classroom learning environment to the e-learning one, because in a traditional classroom students feel they have more opportunities to debate, deliberate, and discuss with their educators and fellows [9]. Other issues related to e-learning are the infrastructure development, institutional problems, availability of technological resources and limited human resources [12]. To have a successful e-learning strategy, it is important to be aware of the broad range of e-learning problems that could have to be faced. Khamparia and Pandley [13,14] classified e-learning problems in seven categories: learning path generation, object recommendation, personalization of content, context learning problem, information retrieval, domain ontology construction, and classification of learning styles. Thus, all of these issues are fruitful insights to consider when we intend to enhance e-learning in multidisciplinary teams in HEIs.

Despite these issues, e-learning is a key component of today’s HEIs; however in several institutions e-learning possibilities are very basic. Al-Ammary et al. [15] reported that e-learning is being used in the majority of the universities mainly for uploading and downloading resources and assignments, which are considered basic services provided by most e-learning platforms. Furthermore, the use of content such as video and innovative applications is still new for many teachers, even at the higher education level in developing countries [16,17,18]. Other institutions, in addition to using the basic functionalities, also use e-learning technologies for online communication and assessment, although these functionalities are not yet as common [15,19].

Thus, in order to implement a real collaborative platform, Edelhauser and Lupu-Dima [11] proposed the two steps. The first is to recruit at least two IT specialists with the main responsibility of managing the learning management system intended for the virtual library and classes. The second step is to provide training options and also specific support for teachers in order to help them upload courses and create virtual classes and online tests.

In this context, it is important to assess how HEIs are able to adopt and improve e-learning processes and systems. The E-Learning Maturity Model enables institutions to evaluate and compare their capacity to develop, deliver, and support e-learning. There are other maturity models, however we will focus on the Capability Maturity Model (CMM) and the E-Learning Maturity Model as examples (EMM).

The Capability Maturity Model (CMM) delivers a conceptual framework conceived for improving the management and also the development of software products which ultimately would lead to the production of a software able to accomplish the desired objectives. In addition, the CMM identifies the typical features of an effective software process. All the issues essential to a successful project in terms of people, technology, and the process are addressed by institutions [20] (p. 4396).

The E-learning Maturity Model (EMM) allows institutions to evaluate and compare their capacity to design, deploy, and support e-learning sustainably (Marshall, no date). The CMM and SPICE (Software Process Improvement and Capability Determination) techniques provide the foundation for the EMM.

The EMM is designed to help institutions improve their effectiveness in any areas of work by providing them with methods and tools that can be replicated and adapted as demand grows [21].

In conclusion, maturity models focus on helping institutions develop the ability to identify their own priorities, guarantee quality standards, and make continual improvements [20].

Adoption and implementation of learning information technology have been the subjects of extensive study in the field of learning technologies. The Technology Acceptance Model (TAM) [22,23] and the Unified Theory of Acceptance and Use of Technology (UTAUT) are two of the most frequently utilized theories in this field [24]. Both TAM and UTAUT show that a person’s behavioral intention to utilize technology influences their actual use of it. In TAM, the planned usage is influenced by the attitude toward employing the technology, which is determined by two system perceptions: perceived usefulness and perceived ease of use. Multiple external factors influence both perspectives. UTAUT is based on TAM and seven additional theoretical frameworks. It offers four components that determine usage intention: performance anticipation, effort anticipation, social influence, and facilitating factors. Age, gender, experience, and willingness to use modulate the influence of expectations and facilitating factors on intention [24].

3. Materials and Methods

Based on the literature review, a survey was created to analyze the current state of digital education and subject-specific teaching, and to perceive the main skills and competencies needed to provide the student with training activities through digital education methods. The survey was of a voluntary response and targeted the main players in the academic environment: students, teachers, and librarians. Hence, this paper analyzes the results of the Portuguese survey regarding these three different profiles.

The English survey, prepared by the Portuguese partner, was translated into Portuguese and prepared for distribution using Limesurvey. The address of the survey was sent to potential respondents by email by the project members, social networks, and several other Portuguese institutions of higher education, between the 7 July and 18 October 2021. Data from each country’s survey were collected by the project partner and analyzed using IBM SPSS. The survey questions were answered on a 1 to 5 Likert scale.

Overall, 392 voluntary respondents accessed the Portuguese survey, but only 231 respondents completed the survey. Most respondents were students, with a valid percentage of 61%, and 26% of respondents were teachers.

Briefly, illustrative descriptions were produced for each country regarding their target audience, gender, and age. The computation of descriptive statistics resulted in charts for each target group and dimension. Comparative boxplots and confidence intervals were examined in charts that grouped the questions for each survey dimension.

The questions were categorized into eight theoretical categories from the literature (

Table 1. Definitions of the theoretical constructs of the research model.

Constructs	Definition
Organizational factors	Sumner and Hostetler [25] classified the organizational elements that may influence the adoption of technology in education as motivators/demotivators. The organizational elements were training, technology alignment, organization-support, and technical support.
Technological factors	The quality of the system, the quality of the information, and the quality of the service assistance might be affected by technological or information system variables [26].
Personal factors	Bandura [27] characterizes self-efficacy as a crucial factor in the acceptance of any information system, including learning management systems. Leidner and Jarvenpaa [28] cite the instructor’s attitude towards e-learning as an additional factor associated with LMS acceptability. According to Venkatesh and Davis [24], experience with the use of technology (EUT) also has a significant impact on the acceptance of technology (2000). Rarely examined but highlighted in the literature, is the instructor’s method of instruction. According to Webster and Hackley [29], an instructor with an engaged teaching style is essential for the achievement of learning objectives. In addition, personal inventiveness is a crucial topic that has lately been highlighted in the e-learning literature.
Perceived ease of use	“the degree to which a person believes that using a particular system would be free from effort” [23] (p. 320)
Perceived usefulness	“the degree to which a person believes that using a particular system would enhance their job performance” [23] (p. 320), [30].
Use intention	The predisposition of a user to adopt a particular technology [22].
Actual use	Level of user knowledge regarding e-learning applications, Ashcroft and Watts [31].
Perceived satisfaction	Level of satisfaction regarding the use of applications, Nielsen [32], Wilkins et al. [33]

), including external elements and user experience. The meaning of each question was analyzed and its importance to these theoretical constructs was identified. Each question could be assigned to one or more construct, according to its sense. After this mapping was made, the scores for the constructs were computed as an average score of the answers (in Likert scales) of the corresponding questions from the questionnaire. The list of questions that were used to compute each construct are presented in

Table A1. Formulas for the constructs.

Construct	Students	Teachers	Librarians
Organizational Factors	S_Organizational = (S02b + S02g + S02h + S04_S04a + S04_S04b + S04_S04c + S04_S04j + S04_S04k + S04_S04l)/9.	T_Organizational = (T01_T01g + T02_T02a + T02_T02j + T03b + T03c + T03d + T03e + T03f + T03g + T03h)/10.	L_Organizational = (L02f + L02g + L05_L05c + L05_L05e + L05_L05i)/5.
Technological Factors	S_Technological = (S01c_inverted + S02a + S02c + S02d + S02e + S02f + S02g + S04_S04d + S04_S04e + S04_S04f + S05_S05a + S05_S05b + S05_S05c + S05_S05d + S05_S05e + S05_S05f + S05_S05g)/17.	T_Technological = (T01_T01g + T02_T02a + T02_T02c + T03a + T03b + T03c + T03d + T03e + T03f + T03g + T03h + T04a + T04b + T04c + T04d + T04e + T04f + T04g + T04h + T04i + T04j + T04k + T04l + T07_T07a + T07_T07b + T07_T07c + T07_T07d + T07_T07e + T07_T07f + T07_T07g)/30.	L_Technological = (L01c_inverted + L02a + L02b + L02c + L02d + L02e + L02f + L03a + L03b + L03c + L03d + L03e + L03f + L03g + L03h + L03i + L03j + L03k + L03l + L04a + L04b + L04c + L04d + L04e + L04f + L04g + L04h + L04i + L04j + L04k + L05_L05d + L05_L05e + L08_L08a + L08_L08b + L08_L08c + L08_L08d + L08_L08e + L08_L08f + L08_L08g)/39.
Personal Factors	S_Personal = (S03_S03a + S03_S03b + S03_S03c + S03_S03d + S03_S03e + S03_S03f + S03_S03g + S03_S03h + S03_S03i + S03_S03j + S03_S03k + S03_S03l + S04_S04h + S04_S04i)/14.	T_Personal = (T02_T02b + T02_T02h + T02_T02k + T02_T02l + T02m_inverted + T06_T06a + T06_T06b + T06_T06c + T06_T06d + T06_T06e + T06_T06f + T06_T06g + T06_T06h + T06_T06i + T06_T06j + T06_T06k + T06_T06l)/17.	L_Personal = (L07_L07a + L07_L07b + L07_L07c + L07_L07d + L07_L07e + L07_L07f + L07_L07g + L07_L07h + L07_L07i + L07_L07j + L07_L07k + L07_L07l)/12.
Perceived Usefulness	S_Usefulness = (S01_S01b + S01_S01d + S01_S01h)/3.	T_Usefulness = (T01_T01d + T01_T01j)/2.	L_Usefulness = (L01_L01b + L01_L01d + L01_L01h + L05_L05a + L05_L05b + L05_L05d + L05_L05e + L05_L05h + L05_L05i)/9.
Perceived ease of use	S_Ease_of_use = (S01_S01b + S01c_inverted + S01_S01g + S04_S04g)/4.	T_Ease_of_use = (T01_T01c + T01_T01g + T01_T01h + T01_T01i)/4.	L_Ease_of_use = (L01_L01b + L01c_inverted + L01_L01g + L05_L05h + L05_L05j)/5.
Intention to use	S_Intention = (S01_S01e + S01_S01f + S01_S01i + S01_S01j)/4	No questions identified in this construct	L_Intention = (L01_L01d + L01_L01e + L01_L01f + L01_L01i + L05_L05a + L05_L05c + L05_L05d + L05_L05f + L05_L05g + L05_L05j)/10.
Actual use	S_Use = (S02a + S02b + S02c + S02d + S02e + S02f + S02g + S02h + S04_S04e + S04_S04f + S04_S04g + S04_S04h + S04_S04i + S04_S04j)/14.	T_Use = (T01_T01a + T02_T02d + T02_T02e + T02_T02f + T02_T02g + T02_T02h + T02_T02i + T04a + T04b + T04c + T04d + T04e + T04f + T04g + T04h + T04i + T04j + T04k + T04l)/19.	L_Use = (L02a + L02b + L02c + L02d + L02e + L02f + L02g + L03a + L03b + L03c + L03d + L03e + L03f + L03g + L03h + L03i + L03j + L03k + L03l + L04a + L04b + L04c + L04d + L04e + L04f + L04g + L04h + L04i + L04j + L04k)/30.
Perceived satisfaction	S_Satisfaction = (S01_S01a + S01_S01b + S01_S01d + S01_S01i + S04_S04g + S04_S04h)/6.	T_Satisfaction = (T01_T01a + T01_T01b + T01_T01c + T01_T01d + T01_T01e + T01_T01f + T01_T01g + T01_T01h + T01_T01i + T01_T01j)/10.	L_Satisfaction = (L01_L01a + L01_L01b + L01_L01i)/3.

. Figure 1 displays the expected relationships between the constructs.

The adopted research model is based on the Technology Adoption Model (TAM) by Davis [22] and the Unified Theory of Acceptance and Application of Technology (UTAUT) by Venkatesh [24]. This conceptual model intends to explain both the behavior and objectives of e-learning system users.

For each construct, Cronbach’s Alpha was calculated to determine the reliability of the questions testing these notions. The Cronbach’s Alpha coefficient is a well-known measure of the internal consistency of a set of variables [34]. The expected correlation between the employed scale and other hypothetical scales with the same number of items in the same universe assesses the same attribute. Values greater than 0.8 suggest good internal consistency, values above 0.9 imply outstanding reliability, and values below 0.6 may indicate that the set of items has poor internal consistency.

A confirmatory factor analysis was also performed to assess the grouping of the questions and to evaluate each question’s contribution to the resulting factors. The principal components method extracted 8 components from the original 49 questions directed to students, retaining 74.8% of the variance; 8 components were extracted from the 68 questions targeted to librarians, preserving 83% of the variance; and 7 components were extracted from the 62 questions made for teachers, explaining 71.9% of the variance. Three orthogonal rotation methods were experimented, with similar results. The communalities and the components matrix for the VARIMAX rotation method are presented in the Appendix B. Most of the communalities have values higher than 0.7, meaning that most of the questions have a good contribution to the resulting factors. On the three groups of factor analysis, there are only 4 questions with communality lower than 0.5, namely S01c, T02b, T02i, and T04i. These questions address training and authenticity of assessment, which are issues that are not consensual among Portuguese teachers and students, but are very important to discuss, therefore these items were kept. The questions most predominant in each factor (

Table A2. Factor analysis of the questions targeted for Students. Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Code	Question	Communalities	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7	Factor 8
S01_S01a	I am satisfied with the elearning experience	0.715	0.523				0.427		0.319
S01_S01b	I am satisfied with the elearning contents/materials provided via e-learning to support learning	0.624	0.378						0.329	0.427
S01_S01c	I have difficulties with elearning	0.254	−0.396
S01_S01d	I believe e-learning is a useful learning option	0.776	0.512			0.413		0.317	0.404
S01_S01e	I intend to use e-learning to assist my own learning	0.772	0.516			0.376		0.306	0.352
S01_S01f	I intend to use e-learning as an autonomous learning tool	0.768	0.352			0.613	0.362
S01_S01g	I believe e-learning can assist the teacher-learner interaction	0.563	0.654
S01_S01h	I believe e-learning can contribute to learning efficiency	0.844	0.753			0.348
S01_S01i	I believe e-learning can contribute to increase learning motivation	0.756	0.773
S01_S01j	I intend to use e-learning in the future	0.790	0.674			0.359			0.387
S02a	e-learning facilities (e.g., computers, projection systems, lecture capture systems, SMART boards, etc.)	0.684			0.683
S02b	Library facilities and services	0.783			0.417			0.353	0.649
S02c	Microsoft office applications or similar (text processor, spreadsheets, databases, presentation applications)	0.703			0.582					−0.375
S02d	Editing tools (multimedia authoring, graphic editing, digital audio and video editing)	0.768			0.805
S02e	ePortfolio	0.802			0.800
S02f	Online or virtual technologies (e.g., network or cloud-based file storage system, Web portals, etc.)	0.809			0.725
S02g	Access to software (e.g., MATLAB, GIS applications, statistical software, qualitative data analysis, graphics software, textual or image analysis program, etc.)	0.844			0.849
S02h	Support for maintenance and repair of ICTs	0.751			0.800
S03_S03a	Communication skills (i.e., writing, verbal)	0.670	0.337				0.670
S03_S03b	Problem-solving ability	0.669	0.513				0.452
S03_S03c	Time management	0.758	0.374				0.734
S03_S03d	Motivation	0.856	0.844
S03_S03e	Work planning and organization	0.723	0.539				0.535
S03_S03f	Desire to learn	0.854	0.886
S03_S03g	Perseverance	0.828	0.855
S03_S03h	Self-confidence	0.718	0.712				0.327
S03_S03i	Self-monitoring	0.777	0.791
S03_S03j	Flexibility	0.706					0.741
S03_S03k	Independency	0.678					0.735
S03_S03l	Working in team and cooperation	0.670	0.495							0.506
S04_S04a	Adjustments to the school calendar	0.663	0.361			0.569
S04_S04b	Level of support regarding the use of the e-learning tools/systems	0.706	0.396			0.424		0.501
S04_S04c	Subsidized/free devices for online/virtual access	0.712						0.761
S04_S04d	Offer/negotiate access to internet at subsidized or zero cost	0.777						0.797
S04_S04e	Use of synchronous tools (Zoom, Teams, Google meets, Skype, others)	0.802				0.787
S04_S04f	Use of asynchronous tools (Moodle, Teams, others)	0.814				0.765
S04_S04g	Suitability of the pedagogical contents provided by the teacher to the online context	0.753			0.342	0.607
S04_S04h	Adequacy of time for synchronous classes	0.788	0.348		0.306	0.482		0.301		0.459
S04_S04i	Relevance of participating in synchronous classes with students personal camera on	0.536						0.507
S04_S04j	Level of support provided by library services	0.864			0.434			0.573	0.449
S04_S04k	Level of psychosocial and emotional support (e.g., chat groups, online forums to share emotions and problems due to COVID-19)	0.732			0.361			0.602	0.334
S04_S04l	I believe that the services and supports provided by my institution during the COVID-19 pandemic were satisfactory	0.716	0.351		0.335	0.485		0.371
S05_S05a	Install and update antivirus software	0.770		0.791
S05_S05b	Install and update spyware software	0.802		0.828
S05_S05c	Definition of authentication profiles	0.862		0.866
S05_S05d	Regular updates of installed software	0.792		0.856
S05_S05e	Adequate use of the firewall	0.880		0.896
S05_S05f	Use of the browser’s security settings	0.893		0.911
S05_S05g	Use of reliable software/open educational resources	0.882		0.906
	Initial Eigenvalues		17.704	5.445	4.159	2.744	2.322	1.483	1.444	1.355
	% of Variance (before rotation)		36.131	11.113	8.489	5.600	4.739	3.026	2.946	2.764
	Rotation Sums of Squared Loadings		8.736	6.195	5.658	4.200	3.988	3.926	2.101	1.853
	% of Variance (after rotation)		17.828	12.644	11.547	8.571	8.138	8.011	4.288	3.781
	Constructs identified		Personal	Technological	Actual use	Usefulness/Intention	Personal	Organizational	Satisfaction	Actual use

,

Table A3. Factor analysis of the questions targeted for Librarians. Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Code	Question	Communalities	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7	Factor 8
L01_L01a	I am satisfied with the e-learning experience	0.844		0.484		0.533	0.318	0.333
L01_L01b	I am satisfied with the e-learning contents and materials provided via e-learning to support learning of students	0.853		0.398		0.505		0.317	0.447
L01_L01c	I have difficulties with e-learning	0.570			−0.341	−0.517
L01_L01d	I believe e-learning is a useful tool for librarians to deliver information literacy and research skills training for students and other library users	0.832	0.323			0.362	0.656	0.365
L01_L01e	I intend to use e-learning to assist library services	0.881				0.557	0.606	0.326
L01_L01f	I intend to use e-learning as an autonomous learning option	0.871				0.658	0.513
L01_L01g	I believe e-learning can assist librarian-library user interaction	0.801				0.862
L01_L01h	I believe e-learning can contribute to learning efficiency	0.880				0.804	0.340
L01_L01i	I believe e-learning can help increase learning motivation	0.882		0.335		0.743
L02a	e-learning facilities (e.g., computers, projection systems, lecture capture systems, SMART boards, etc.)	0.730						0.727
L02b	Microsoft office applications or similar (text processor, spreadsheets, databases, presentation applications)	0.849				0.312		0.798
L02c	Editing tools (multimedia authoring, graphic editing, digital audio and video editing)	0.905	0.375			0.311		0.543	0.518
L02d	ePortfolio	0.870	0.331					0.517	0.495	0.356
L02e	Online or virtual technologies (e.g., network or cloud-based file storage system, Web portals, etc.)	0.787					0.312	0.310		0.614
L02f	Access to software (e.g., MATLAB, GIS applications, statistical software, qualitative data analysis, graphics software, textual or image analysis program, etc.)	0.781						0.851
L02g	Support for maintenance and repair of ICTs	0.874		0.463				0.587		0.502
L03a	Images (pictures, photographs, including from the Web)	0.866				0.338	0.641	0.496
L03b	Presentations (e.g., PowerPoint, including from online sources)	0.873					0.609	0.618
L03c	Word files (activity sheets/handouts/notes)	0.620	0.432					0.574
L03d	Digital films/video (e.g., from YouTube)	0.773	−0.463				0.449	0.423
L03e	Online collaboration tools (e.g., Adobe Connect, Google Docs)	0.729			0.478		0.466		0.360
L03f	ePortfolio	0.823			0.341	0.322			0.690
L03g	eBooks/eTextbooks	0.732	0.480						0.526	0.328
L03h	Educational games/simulations	0.594		0.337					0.601
L03i	Lecture capture tools	0.828	−0.473	0.393			0.318	0.524
L03j	Accessible tools (for people with disabilities)	0.611		0.349					0.390	−0.331
L03k	Web 2.0 tools (wikis, blogs, social networking and sharing tools)	0.868				0.416	0.531		0.539
L03l	Learning objects (Scorms/IMS content)	0.811		0.341					0.758
L04a	OER Commons	0.852	0.855
L04b	Saylor Academy	0.970	0.927
L04c	WikiEducator	0.921	0.910
L04d	OpenStax College	0.972	0.952
L04e	BC Campus Open Textbooks	0.928	0.933
L04f	NPTEL, India	0.952	0.959
L04g	MIT Open Courseware	0.892	0.825							0.362
L04h	OpenLearn, UK	0.972	0.952
L04i	CollegeOpenTextbook	0.858	0.745							0.478
L04j	Directory of Open Access Journals	0.748	0.577							0.554
L04k	MERLOT	0.834	0.664						0.487
L05_L05a	Librarians working as instructors of technologies to support students and teachers	0.694				0.760
L05_L05b	Librarians delivering information literacy and research skills training for students and other library users	0.804				0.387	0.729
L05_L05c	Libraries should manage or support the management of the e-learning infrastructure	0.669				0.547				0.381
L05_L05d	Libraries should have a specific module integrated on e-learning management systems to gain visibility among students	0.770			0.498		0.553	−0.346
L05_L05e	Libraries available 24/7 with online reference services	0.816						−0.447	0.508
L05_L05f	Libraries should prepare online tutorials for resources access	0.761					0.760
L05_L05g	Libraries should prepare online or blended training	0.792	−0.400	0.347			0.651
L05_L05h	The library website or catalog should make available educational resources prepared by academic staff to support e-learning (PPT, and online tutorials, etc.)	0.668			0.350	0.367	0.610
L05_L05i	The library should manage an e-learning repository	0.786		0.432						0.653
L05_L05j	Libraries should endeavor to reach students who do not attend the academic library space	0.674		0.351			0.647
L07_L07a	Communication skills (i.e., writing, verbal)	0.761		0.729
L07_L07b	Problem-solving ability	0.792		0.425	0.339				−0.622
L07_L07c	Time management	0.884		0.689		0.484
L07_L07d	Motivation	0.950		0.822		0.384
L07_L07e	Work Planning and organization	0.834		0.693		0.424
L07_L07f	Desire to learn	0.889		0.768		0.304	0.306
L07_L07g	Perseverance	0.947		0.887
L07_L07h	Self-confidence	0.935		0.883
L07_L07i	Self-monitoring	0.909		0.893
L07_L07j	Flexibility	0.840		0.697	0.438
L07_L07k	Independency	0.818	0.474	0.383			0.414	0.434
L07_L07l	Team work and cooperation	0.911		0.649		0.501			0.315
L08_L08a	Install and update antivirus software	0.961			0.897
L08_L08b	Install and update spyware software	0.936		0.418	0.831
L08_L08c	Definition of authentication profiles	0.827			0.757				0.432
L08_L08d	Regular updates of installed software	0.917			0.855
L08_L08e	Adequate use of the firewall	0.873			0.831
L08_L08f	Use of the browser’s security settings	0.950			0.898
L08_L08g	Use of reliable software/open educational resources	0.919		0.315	0.890
	Initial Eigenvalues		19.370	12.795	6.771	4.754	4.103	3.976	2.482	2.275
	% of Variance (before rotation)		28.486	18.816	9.957	6.991	6.034	5.847	3.651	3.346
	Rotation Sums of Squared Loadings		10.979	9.773	7.870	6.888	6.653	6.209	5.025	3.129
	% of Variance (after rotation)		16.146	14.373	11.574	10.129	9.783	9.132	7.389	4.602
	Constructs identified		Actual use	Satisfaction/Personal	Technological	Intention/Ease of use	Usefulness	Actual use	Actual use	Organizational

and

Table A4. Factor analysis of the questions targeted for Teachers. Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Code	Question	Communalities	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7
T01_T01a	Flexibility	0.648	0.568			0.387		−0.305
T01_T01b	Wide range of tools	0.790				0.810
T01_T01c	Ease of use	0.715	0.475			0.424			−0.457
T01_T01d	Usefulness	0.676				0.719
T01_T01e	Customization (ability to personalize learning for students)	0.763		0.449		0.608
T01_T01f	Innovation (i.e., freedom to experiment with teaching practice)	0.827	0.302			0.812
T01_T01g	Accessibility (platforms, materials, resources)	0.798	0.359			0.748
T01_T01h	Increases engagement and enjoyment for students	0.720		0.444		0.609
T01_T01i	An improved relationship with students	0.700		0.450	0.300	0.579
T01_T01j	Increased autonomy, motivation, self-determination and self-regulation	0.781		0.356		0.749
T02_T02a	Teachers’ access to technology (computers, software, stable internet connection)	0.509					0.609
T02_T02b	Lack of training to deliver education in an online environment	0.395				0.491
T02_T02c	Students’ access to technology	0.772	0.368	0.316	0.483		0.458
T02_T02d	Communicating with students	0.719			−0.309		0.717
T02_T02e	Involving students	0.737			−0.487		0.631
T02_T02f	Keeping students motivated and engaged	0.883			−0.615		0.648
T02_T02g	Supporting students with special needs or disabilities	0.757			−0.548		0.579
T02_T02h	Converting activities and content for use in e-learning	0.500			−0.411	0.315	0.457
T02_T02i	Authentically assessing students’ progress	0.489					0.609
T02_T02j	Availability of clear guidelines regarding online learning from the school board	0.634				0.531	0.405		0.382
T02_T02k	Increased workload and stress working from home	0.833					0.847
T02_T02l	Time management and organization	0.690					0.814
T02_T02m	There have been no challenges	0.620	−0.369		0.391		−0.506
T03a	e-learning facilities (e.g., computers, projection systems, lecture capture systems, SMART boards, etc.)	0.607		0.315	0.316			0.597
T03b	Library facilities and services	0.608						0.400	−0.594
T03c	Microsoft office applications or similar (text processor, spreadsheets, databases, presentation applications)	0.522	0.400					0.506
T03d	Editing tools (Multimedia authoring, Graphic editing, Digital audio and Video editing)	0.774				0.383		0.724
T03e	ePortfolio	0.786					−0.378	0.724
T03f	Online or virtual technologies (e.g., network or cloud-based file storage system, Web portals, etc.)	0.736	0.416	0.324				0.612
T03g	Access to software (e.g., MATLAB, GIS applications, statistical software, qualitative data analysis, graphics software, textual or image analysis program, etc.)	0.808					−0.348	0.750
T03h	Support in the maintenance and repair of ICTs	0.578						0.472	−0.549
T04a	Images (pictures, photographs, including from the Web)	0.723	0.411	0.333					0.569
T04b	Presentations (e.g., PowerPoint, including from online sources)	0.733			−0.688
T04c	Word files (activity sheets/handouts/notes)	0.643			−0.538			0.485
T04d	Digital films/video (e.g., from YouTube)	0.724				0.323			0.729
T04e	Online collaboration tools (e.g., Adobe Connect, Google Docs)	0.650	0.451		0.439			0.331	0.365
T04f	ePortfolio	0.668			0.687
T04g	eBooks/eTextbooks	0.581			0.380				0.495
T04h	Educational games/simulations	0.695			0.800
T04i	Lecture capture tools	0.230							0.333
T04j	Accessible tools (for people with disabilities)	0.866			0.844
T04k	Web 2.0 tools (wikis, blogs, social networking and sharing tools)	0.624		0.320	0.646
T04l	Learning objects (Scorms/IMS content)	0.823			0.845
T06_T06a	Communication skills (i.e., writing, verbal)	0.732	0.718		−0.330		0.305
T06_T06b	Problem-solving ability	0.722	0.752
T06_T06c	Time management	0.559	0.501		−0.393
T06_T06d	Motivation	0.719	0.760
T06_T06e	Work Planning & organization	0.803	0.852
T06_T06f	Desire to learn	0.796	0.823
T06_T06g	Perseverance	0.772	0.798
T06_T06h	Self-confidence	0.879	0.827	0.397
T06_T06i	Self-monitoring	0.819	0.824	0.304
T06_T06j	Flexibility	0.906	0.872	0.303
T06_T06k	Independency	0.713	0.786
T06_T06l	Team work and cooperation	0.701	0.672					0.396
T07_T07a	Install and update antivirus software	0.822		0.817
T07_T07b	Install and update spyware software	0.885		0.894
T07_T07c	Definition of authentication profiles	0.840		0.874
T07_T07d	Regular updates of installed software	0.908		0.916
T07_T07e	Adequate use of the firewall	0.906		0.868		0.317
T07_T07f	Use of the browser’s security settings	0.893		0.885
T07_T07g	Use of reliable software/open educational resources	0.855		0.888
	Initial Eigenvalues		15.171	10.244	5.053	4.530	3.729	3.270	2.564
	% of Variance (before rotation)		24.469	16.523	8.150	7.307	6.015	5.274	4.136
	Rotation Sums of Squared Loadings		9.775	7.841	6.959	6.271	5.686	4.537	3.494
	% of Variance (after rotation)		15.766	12.646	11.224	10.114	9.171	7.318	5.635
	Constructs identified		Personal/Ease of use	Satisfaction	Technological	Usefulness	Actual use	Organizational	Actual use

) lead us to the challenge of mapping the factors to the theoretical constructs addressed in this paper. However, in all rotation methods, there are still questions that overlap in several factors, and factors that can be identified with more than one construct.

Therefore, we decided to proceed with the work of analyzing the constructs based on the TAM and UTAUT methodologies, measured by the average scores of the corresponding questions (as in Table A1) because they are more interpretable than the factors obtained with the factor analysis and suit the purpose of this research best.

The research hypotheses for this work were defined using the constructs defined in Table 1, following the scheme shown in Figure 1:

H1:

External factors have a positive impact on the user experience.

H1a: Organizational factors can increase the use of e-learning tools.

H1b: Technological factors can increase the use of e-learning tools.

H1c: Personal factors can reduce the use of e-learning tools.

H2:

User experience has a positive impact on the perceived satisfaction.

H2a: Perceived usefulness positively influences the perceived satisfaction.

H2b: Perceived ease of use positively influences the perceived satisfaction.

To evaluate these research hypotheses, correlations between constructs were examined, and linear regressions were estimated. The residuals of all regression analyses were assessed for normality, and no substantial deviations were found.

4. Results

In the sample, 64.5% of the respondents were female, 34.5% were male, one student identified as non-binary, and one student declined to respond. A total of 60 teachers responded to the study, with 65% of respondents female, 31.7% male, and two teachers opting not to respond.

The bulk of respondents were between 21 and 49 years old, and the average age was 35. The median age of the students polled was approximately 22 years, whereas the median age of the teachers and librarians was approximately 50 years.

In response to the question “What device(s) do you use most frequently for e-learning?”, most respondents answered laptops, followed by cellphones and desktop computers. Note that students utilize smartphones for e-learning considerably more than other players (Figure 2).

4.1. Results for the Portuguese Students

Cronbach’s alpha was used to evaluate the reliability of the survey and the components. Regarding the reliability analysis of the constructs for the student group (

Table 2. Reliability analysis of the constructs for students.

Construct	Nb. of Items	Cronbach’s Alpha
Organizational factors	9	0.885
Technological factors	17	0.908
Personal factors	14	0.920
Perceived usefulness	3	0.833
Perceived ease of use	4	0.688
Use intention	4	0.878
Actual use	14	0.920
Perceived satisfaction	6	0.886

), the statistics revealed a high level (excellent and good) for most of the constructs, except for perceived ease of use, which obtained 0.688, a low level of internal consistency between items; however, a reliability coefficient of 0.70 or higher is considered “acceptable” in the majority of social science research situations. The majority of the remaining structures were rated good and exceptional.

Almost all constructions produced for the student’s data are internally consistent (or excellent). Only the perceived ease of use construct has a moderate Cronbach’s alpha due to the limited number of components within this construct.

The majority of students awarded good ratings (greater than 3) to all components, placing personal factors and intention to use at the highest level, while organizational factors and actual use contributed to the opposite position, a low ranking (

Table 3. Descriptive statistics regarding the constructs for students.

	Mean	Median	Std. Deviation
Perceived ease of use	3.4167	3.5000	0.77571
Intention to use	3.5128	3.5000	0.98684
Organizational factors	3.1054	3.1111	0.82856
Personal factors	3.6081	3.6429	0.75669
Perceived satisfaction	3.3910	3.3333	0.87898
Technological factors	3.4721	3.4412	0.71449
Actual use	3.3755	3.3571	0.76676
Perceived usefulness	3.4017	3.6667	0.96154

).

The Pearson correlation coefficient (

Table 4. Pearson correlation coefficient between constructs for students.

Pearson Correlation Coefficient (Students)	Perceived Ease of Use	Intention to Use	Organizational Factors	Personal Factors	Perceived Satisfaction	Technological Factors	Actual Use	Perceived Usefulness
Perceived ease of use	1
Intention to use	0.754 **	1
Organizational factors	0.573 **	0.559 **	1
Personal factors	0.710 **	0.720 **	0.478 **	1
Perceived satisfaction	0.879 **	0.832 **	0.661 **	0.779 **	1
Technological factors	0.497 **	0.485 **	0.689 **	0.522 **	0.531 **	1
Actual use	0.605 **	0.592 **	0.875 **	0.571 **	0.693 **	0.860 **	1
Perceived fsefulness	0.840 **	0.872 **	0.609 **	0.736 **	0.934 **	0.423 **	0.587 **	1

**. Correlation is significant at the 0.01 level (2-tailed).

) demonstrates a strong link between perceived satisfaction and perceived usefulness (0.934). Moreover, perceived satisfaction and perceived ease of use exhibit a positive linear association of 0.877. In addition, perceived satisfaction and personal variables, and perceived satisfaction and actual use also have a positive correlation that is not as strong.

Based on the students’ data, a linear regression model for perceived satisfaction based on the other components was estimated. However, not all constructs directly affect satisfaction, and VIF values are significantly high. Student satisfaction appears to be solely determined by personal factors, perceived usefulness, and actual use. Consequently, a regression model containing only the significant variables was computed (Equation (1) and

Table 5. Linear regression model for perceived satisfaction of students.

Dependent Variable: Perceived Satisfaction (Students)	Β	Std. Error	t	p-Value	VIF
(Constant)	−0.184	0.163	−1.126	0.264
Personal factors	0.178	0.063	2.841	0.006	2.399
Actual use	0.219	0.051	4.302	<0.0005	1.625
Perceived usefulness	0.645	0.050	12.894	<0.0005	2.468
R2	0.910
R2Adj	0.907
F	250.220	p < 0.0005

).

$$Satisfaction={\beta }_0+{\beta }_{1}Personal+{\beta }_{2}Use+{\beta }_{3}Usefulness+\epsilon$$

(1)

The model correctly explains 91% of the variance in satisfaction using these three constructs (R² = 0.910, R²_adj = 0.907), with 9% of the variance in satisfaction due to other factors. The residual analysis validates the model in terms of normality of the residuals and homoscedasticity.

A model for the construct actual use, depending on the remaining constructs was estimated. Only the organizational factors and the technological factors are significant in predicting the actual use. This model is in Equation (2) and the coefficients are in

Table 6. Linear regression model for actual use of e-learning by students.

Dependent Variable: Actual Use (Students)	Β	Std. Error	T	p-Value	VIF
(Constant)	0.007	0.146	0.048	0.961
Organizational factors	0.498	0.049	10.242	<0.0005	1.903
Technological factors	0.525	0.056	9.305	<0.0005	1.903
R2	0.891
R2Adj	0.888
F	307.101	p < 0.0005

.

$$Use={\beta }_0+{\beta }_{1}Organizational+{\beta }_{2}Technological+\epsilon$$

(2)

The model accurately accounts for 89.1% of the variance in actual use, using two dimensions organizational factors and technological factors (R² = 0.891, R²_adj = 0.888). The normality of the residuals and the homoscedasticity assumption was confirmed.

4.2. Results for the Portuguese Teachers

When questioned about the reasons to be pleased with using e-learning environments, the highest answer from teachers was “flexibility” with a mean of 3.97%, followed by “utility” and “diversity of tools” with identical values of 3.90% and 3.83%, respectively. Most instructors consistently rated “ease of use” with the lowest dispersion data in terms of the mean, indicating the most significant consensus (

Table 7. Descriptive statistics regarding the constructs for teachers.

	Mean	Median	Std. Deviation
Perceived ease of use	3.1288	3.2500	0.87972
Organizational factors	3.2030	3.2000	0.64492
Personal factors	3.8467	3.8824	0.67068
Perceived satisfaction	3.3848	3.5000	0.84006
Technological factors	2.9677	2.9333	0.60556
Actual use	3.1994	3.2105	0.42203
Perceived usefulness	3.3485	3.5000	0.94773

).

Regarding the analysis of the research hypothesis regarding the constructs, most teachers provided good ratings (more than 3) to all components, with personal factors and perceived satisfaction receiving the highest scores. On the other hand, technological factors accounted for the opposite position, a low ranking (Table 7).

Regarding the constructs’ reliability analysis (

Table 8. Reliability analysis of the constructs for teachers.

Construct	Nb. of Items	Cronbach’s Alpha
Organizational factors	10	0.720
Technological factors	30	0.882
Personal factors	17	0.884
Perceived usefulness	2	0.721
Perceived ease of use	4	0.814
Actual use	19	0.622
Perceived satisfaction	10	0.928

), the statistics indicated a high level (excellent and good) for perceived satisfaction, personal variables, technological factors, and perceived ease of use. Actual use obtained 0.622, indicating a moderate degree of internal consistency. Note that only seven constructs were computed for the teachers’ data, as there was no intention to use question in the survey. This occurred inadvertently, possibly because teachers were already compelled to use due to the pandemic. Hence, elaborating on their intent to use was no longer a problem.

The Pearson correlation coefficient displays a substantial magnitude (0.944) of association between perceived satisfaction and perceived ease of use (

Table 9. Pearson correlation coefficients between constructs for teachers.

Pearson Correlation Coefficient (Teachers)	Perceived Ease of Use	Organizational Factors	Personal Factors	Perceived Satisfaction	Technological Factors	Actual Use	Perceived Usefulness
Perceived ease of use	1
Organizational factors	0.572 **	1
Personal factors	0.294 *	0.349 *	1
Perceived satisfaction	0.944 **	0.593 **	0.385 **	1
Technological factors	0.682 **	0.715 **	0.399 *	0.599 **	1
Actual use	0.215	0.354 *	0.476 **	0.255 *	0.587 **	1
Perceived usefulness	0.808 **	0.542 **	0.325 *	0.903 **	0.474 **	0.201	1

*. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed)

). In addition, perceived satisfaction and perceived usefulness archive a positive linear correlation of 0.903.

A linear regression model for the perceived satisfaction of teachers was estimated depending on the other constructs. The most significant variables found, at a 5% level, were, as expected, the usefulness and the ease of use (Equation (4)).

$$Satisfaction={\beta }_0+{\beta }_{1}EaseOfUse+{\beta }_{2}Usefulness+\epsilon$$

(3)

As seen in

Table 10. Linear regression model for perceived satisfaction of teachers.

Dependent Variable: Perceived Satisfaction (Teachers)	Β	Std. Error	T	p-Value	VIF
(Constant)	0.239	0.103	2.315	0.024
Perceived ease of use	0.613	0.051	11.908	<0.0005	2.886
Perceived usefulness	0.360	0.046	7.784	<0.0005	2.886
R2	0.947
R2Adj	0.945
F	508.393	p < 0.0005

, the model correctly explains 94.7% of the variance in perceived satisfaction using the two constructs perceived ease of use and perceived usefulness (R² = 0.947, R²_adj = 0.945). The residuals seem to be homoscedastic and approximately normally distributed.

The model in Equation (4) is a linear regression model for the construct actual use of the teachers, with only the significant variables at a 5% level.

$$Use={\beta }_0+{\beta }_{1}Personal+{\beta }_{2}Technological+\epsilon$$

(4)

The coefficients of this model are in

Table 11. Linear regression model for actual use of e-learning by teachers.

Dependent Variable: Actual Use (Teachers)	Β	Std. Error	T	p-Value	VIF
(Constant)	1.181	0.332	3.560	0.001
Personal factors	0.317	0.083	3.810	0.001	1.190
Technological factors	0.269	0.092	2.916	0.007	1.190
R2	0.558
R2Adj	0.529
F	18.972	p < 0.0005

. The linear model correctly explains only 55.8% of the variance in actual use from the personal and technological factors (R² = 0.558, R²_adj = 0.529), and so 44.2% of the variance in actual use may be explained by other factors not included in this model. The residuals seem to be homoscedastic and not very different from a normal distribution.

4.3. Results for the Portuguese Librarians

The reliability of the constructs for the librarian’s group is presented in

Table 12. Reliability analysis of the constructs for librarians.

Construct	Nb. of Items	Cronbach’s Alpha
Organizational factors	5	0.567
Technological factors	39	0.929
Personal factors	12	0.953
Perceived usefulness	9	0.865
Perceived ease of use	5	0.795
Use intention	10	0.939
Actual use	30	0.920
Perceived satisfaction	3	0.855

. All constructs present good reliability (near or above 0.8), except for the construct organizational factors which has a very low Cronbach’s alpha (0.567) which means that this construct does not have a good internal consistency.

Librarians scored higher mean values in the constructs intention to use, usefulness, and personal factors. The lowest scores for librarians were observed in technological factors and actual use (

Table 13. Descriptive statistics regarding the constructs for librarians.

	Mean	Median	Std. Deviation
Perceived ease of use	4.0000	4.0000	0.72768
Intention to use	4.1955	4.3500	0.57444
Organizational factors	3.5909	3.8000	0.58709
Personal factors	4.0985	4.0833	0.61001
Perceived satisfaction	3.7727	4.0000	0.99409
Technological factors	2.6783	2.7436	0.55157
Actual use	2.4894	2.4667	0.57066
Perceived usefulness	4.0505	4.2222	0.60249

).

In

Table 14. Pearson correlations between the constructs for librarians.

Pearson Correlation Coefficient (Librarians)	Perceived Ease of Use	Intention to Use	Organizational Factors	Personal Factors	Perceived Satisfaction	Technological Factors	Actual Use	Perceived Usefulness
Perceived ease of use	1
Intention to use	0.862 **	1
Organizational factors	0.573 **	0.637 **	1
Personal factors	0.749 **	0.706 **	0.669 **	1
Perceived satisfaction	0.757 **	0.671 **	0.625 **	0.625 **	1
Technological factors	0.496 *	0.410	0.573 **	0.411	0.355	1
Actual use	0.353	0.352	0.551 **	0.217	0.343	0.931 **	1
Perceived usefulness	0.881 **	0.957 **	0.726 **	0.733 **	0.745 **	0.452 *	0.383	1

*. Correlation is significant at the 0.05 level (2-tailed). **. Correlation is significant at the 0.01 level (2-tailed).

, the Pearson correlations between the constructs developed for librarians are presented. Note the strong correlations between technological factors and the actual use (R = 0.931) and between perceived usefulness and the intention to use (0.957).

A linear regression model for the perceived satisfaction of librarians was estimated with the other constructs. The most significant variables found, at a 10% level, were the usefulness, the ease of use, the actual use, and the technological factors (Equation (5)).

$$Satisfaction={\beta }_0+{\beta }_{1}Technological+{\beta }_{2}EaseOfUse+{\beta }_{3}Use+{\beta }_{4}Usefulness+\epsilon$$

(5)

According to

Table 15. Linear regression model for perceived satisfaction of librarians.

Dependent Variable: Perceived Satisfaction (Librarians)	Β	Std. Error	T	p-Value	VIF
(Constant)	−1.531	0.819	−1.868	0.079
Perceived ease of use	0.766	0.324	2.360	0.030	4.222
Technological factors	−1.336	0.668	−1.999	0.062	10.295
Actual use	1.160	0.607	1.913	0.073	9.083
Perceived usefulness	0.723	0.361	2.003	0.061	3.583
R2	0.773
R2Adj	0.720
F	14.469	p < 0.0005

, note that Technological Factors have a significant negative impact on perceived satisfaction. The model correctly explains 77.3% of the variance in perceived satisfaction using these four constructs (R² = 0.773, R²_adj = 0.720), leaving 22.7% of the variance in librarians’ satisfaction due to other factors not included in this model. From observation, the residuals seem to be homoscedastic and approximately normally distributed.

The model for the construct Actual Use, with only the significant variables at 10% of significance, resulted in the following Equation (6):

$$Use={\beta }_0+{\beta }_{1}Organizational+{\beta }_{2}Personal+{\beta }_{3}Technological+\epsilon$$

(6)

The estimated coefficients are shown in

Table 16. Linear regression model for actual use of e-learning by librarians.

Dependent Variable: Actual Use (Librarians)	Β	Std. Error	T	p-Value	VIF
(Constant)	0.325	0.282	1.153	0.264
Organizational factors	0.195	0.094	2.065	0.054	2.074
Personal factors	−0.276	0.082	−3.383	0.003	1.675
Technological factors	0.969	0.085	11.356	0.000	1.499
R2	0.918
R2Adj	0.905
F	67.409	p < 0.0005

. Note the negative impact of the personal factors on the actual use by librarians, and the positive impact of the technological factors on the actual use. The model correctly explains 91.8% of the variance in actual use using these three constructs (R² = 0.918, R²_adj = 0.905), and the residuals seem to be homoscedastic and approximately normally distributed.

5. Discussion

The survey was fully answered by 141 Portuguese students, 60 teachers, and 30 librarians, with most respondents being female (67.5%). The median age of the polled students was approximately 22 years old, whereas the median age of the teachers and librarians was approximately 50 years old. Students in Portugal had a positive experience with e-learning, expressing minor obstacles in adapting to e-learning and a strong intent to use e-learning in the future.

The most frequently cited reasons for teacher satisfaction with e-learning environments were flexibility, utility, and a variety of tools. In contrast, the least frequently cited reasons were the interaction with students and students’ involvement and satisfaction. Due to working from home, teachers face an increased workload and stress, which is the most commonly mentioned difficulty.

According to the statistics, only 36.7% of the teachers received training on the use of ICTs for teaching and learning. The laptop is the most popular device for e-learning across all target groups, followed by the smartphone among students.

The majority of students and teachers have extensive familiarity with Microsoft Office and similar applications but have limited access to specialized software such as Matlab, GIS, and statistical tools. Students in Portugal place the highest emphasis on adaptability and independence, while teachers place the highest value on communication skills, work planning, and organization.

The Portuguese have a moderate understanding of Internet data security issues, with only 19.1% of students and 18.3% of teachers receiving training in this area, but fortunately a low incidence of cybercrime (9.2% of students, 11.7% of teachers).

Cronbach’s alpha was used to examine the survey and constructs’ reliability, with all constructs reporting good or exceptional reliability. Between the constructs, regression models were constructed independently for each target group.

Among Portuguese students, 91% of e-learning satisfaction is able to be attributed to perceived usefulness, actual use, and personal factors. For educators, satisfaction appears to be mostly dependent on perceived usefulness and ease of use (R² = 0.947). Portuguese librarians’ satisfaction can be positively credited to experience-based factors, such as ease of use, usefulness, and actual use, and also negatively to external technological factors. The actual use of e-learning tools by students is 89% based on organizational and technological variables, but the actual use by teachers appears to be mostly dependent on personal and technological factors. Librarians’ actual use is 92% dependent on technological factors and organizational and personal factors, with the personal factors negatively affecting the actual use by librarians.

With the regression models, the use of e-learning tools was proven to be dependent on the external factors (confirming H1), namely the organizational factors for students and librarians (H1a) and personal factors for teachers and librarians (H1c). These three target groups showed positive relationships between technological factors and the use of e-learning tools (H1b). Usefulness of the online tools appeared as a significant variable with positive impact on perceived satisfaction for all the three players (H2a). Ease of use of the online tools was only significant for attaining a higher satisfaction within the groups of teachers and librarians (H2b).

It was concluded that, to improve e-learning in HEIs, a significant involvement of the institution’s management is necessary, with the purpose of promoting the existence of multidisciplinary teams for the production of reusable digital content.

Each one of the players, according to their profile, would apply their skills and scientific knowledge. Thus, teachers are responsible for contributing within their scientific area of expertise and librarians are responsible for managing the learning objects repositories, metadata processing, copyright management, and international certification. This is important to promote the reuse of the materials by other teachers.

Students, as consumers, are responsible for evaluating the effectiveness of the content in their learning process and promoting changes in a constant cycle of continuous improvement.

To sum up, the team that is involved in the production, archive, distribution, and use of the online learning objects must be interdisciplinary.

6. Conclusions

The present paper reviewed the state-of-the-art of e-learning in Portugal through a quantitative analysis. This analysis aims to develop a conceptual model to explain users’ behavior and intentions when using e-learning systems, identifying the skill shortages and mismatches regarding the readiness to teach in an online environment.

The methodology chosen was the application of an online survey, targeted for the main players in the academic environment (students, teachers and librarians), which contained questions to explain both the behavior and objectives of e-learning system users. The questions were grouped into the constructs of the adopted research model, which is based on the Technology Adoption Model (TAM) by Davis [22] and the Unified Theory of Acceptance and Application of Technology (UTAUT) by Venkatesh [24]. The constructs were combined in linear regression models that can identify the determinants of perceived satisfaction and actual use of e-learning.

The analysis of the survey’s data enabled us to determine the organizational, sociocultural, and technological context elements for evaluating the sustainability of e-learning. In addition, it allowed the evaluation of perceived usefulness, perceived ease of use, intention to use, actual use, and perceived satisfaction. Regarding e-learning use, we conclude that, in general, all participants use e-learning and are pleased with the outcome. Additionally, the personal, technological, and organizational components of e-learning use are recognized.

The innovation of the study is addressing this subject with an integrated vision of three different players in higher education: students, teachers, and librarians. The role of university libraries is crucial in this study, as they will have to adapt their services and provide digital learning materials as well as information and digital skills training, both to teaching staff and students.

This study had some limitations. The sample was a random sample obtained by convenience, which may lack representativeness of the Portuguese population. However, the number of complete answers forms a large sample of people from different institutions, ages, and backgrounds. The number of questions in each construct is variable and the Cronbach’s alpha may be influenced by it. There are questions that were identified with more than one construct (for example, L02f, S02g, T03g were assigned to both organizational and technological factors), which may lead to a natural correlation between the constructs.

As future work, we will extend the analysis of the state of the art in regard to e-learning implementation across all the partner countries of the Digitools project, leading to a multicultural study, and design a guide for best practices in digital education that will be adaptable to a wide range of subject-specific teaching and in multicultural contexts.