Investigating the Impact of COVID-19 on E-Learning: Country Development and COVID-19 Response

Abstract

Due to its severity, the outbreak of COVID-19 led to unprecedented levels of social isolation that affected educational institutions, among others. Digital technologies such as cloud computing and video broadcasting helped the adoption of e-learning during the crisis. However, the speed and efficiency of e-learning adoption during the COVID-19 period varied across countries. This paper compares the adoption of e-learning in European countries before and during the COVID-19 pandemic and the relationship between the pandemic, e-learning, and economic development. First, the adoption of e-learning in European countries before and during the pandemic is compared. Second, using fuzzy C-means clustering, homogeneous groups of European countries are formed based on e-learning indicators for the periods before and during the pandemic. Third, GDP per capita is used as an indicator of economic development and severity indices are used as an indicator of the severity of the response to the pandemic to compare the different clusters. The research results show that economically and digitally advanced countries led the adoption of e-learning in both the period before and the period during the pandemic. However, they also responded less strictly to the pandemic. Less-advanced countries responded more strictly to the pandemic, likely due to a lack of healthcare resources, and also fell behind in the adoption of e-learning.

1. Introduction

In the past, the adoption of new technologies at the national level was mainly driven by economic development. Digitization has boomed in recent decades but not for everyone at the same time, indicating a digital divide [1,2], which refers to the gap between different demographic groups and countries/regions based on their access to modern information and communication technologies [3].

Countries that had insufficient funds for the early development of novel infrastructure such as broadband fell behind. European countries were no exception. The harsh difference between the more developed and the less-developed member countries of the European Union (EU) was evident in the first decade of the 21st century [4]. To overcome this gap, the European Commission introduced several initiatives. It started to measure digital and social development using different indicators under the umbrella term, the DESI index (The Digital Economy and Society Index), but this did not provide the planned outcome of the equal digital development of EU countries [5].

When the COVID-19 pandemic hit, isolation occurred at an invisible level, leading to massive closures of businesses, schools, universities, and public transportation. E-learning, e-commerce, e-business, and e-government became the primary online venues for conducting activities that previously required a physical presence. Based on previous findings, wealthier countries adopted new technologies faster and it was assumed that the same phenomenon would occur with the adoption of e-learning during the pandemic [6]. Therefore, the question arose as to whether this was also the case in European countries.

Considering that European countries did not uniformly adopt e-learning during the pandemic, this paper aims to shed light on the relationship between the pandemic, e-learning, and economic development. To this end, we compare a sample of European countries in two years, 2017 (pre-COVID-19) and 2020 (during COVID-19).

To investigate the relationship between the pandemic, e-learning, and economic development, we pose three research questions: (i) RQ1—How did the level of e-learning change in European countries during the pandemic compared to before the pandemic? (ii) RQ2—How homogeneous was the adoption of e-learning in European countries during the pandemic? and (iii) RQ3—What was the impact of GDP per capita on e-learning before and during the pandemic?

To address the above questions, we compare e-learning levels and economic and COVID-19 indicators before and during the pandemic in European countries using fuzzy C-means clustering, ANOVA, and post hoc methods.

This article is organized as follows. Section 2 details the research methodology. In the Section 3, the empirical results are presented using descriptive statistics and the results of the fuzzy C-means clustering analysis. Further analyses are performed using ANOVA. In the Section 5, the concluding remarks, limitations, and recommendations for further research are presented.

2. Background

2.1. The Notion of E-Learning

Digital technologies, such as cloud computing, augmented reality, artificial intelligence, and video broadcasting, have transformed teaching from a traditional approach into a more contemporary and modernized approach [7,8]. E-learning falls under the broader concept of technology-based learning, which is based on innovations in digital transformation [9].

E-learning, the platform for delivering educational content and facilitating learning through technology, has recently gained popularity due to its convenience and adaptability. In contrast, traditional classrooms still use “chalk and lecture” and conventional face-to-face teaching methods in which a teacher writes on a blackboard and speaks to a group of students. Both approaches have advantages and disadvantages and the decision about which one to use depends on variables such as the subject matter, learning objectives, and available resources [10,11]. The “chalk and talk” method is preferable for subjects that focus on practical applications and are more difficult to teach using e-learning methods via an online platform [12]. The combination of both methods seems to be a reasonable option for most subjects in many cases [13].

2.2. E-Learning during the COVID-19 Era

National boundaries did not constrain the COVID-19 pandemic; therefore, it affected people of all nationalities, education and income levels, and genders. COVID-19 resulted in nationwide school closures in most countries, most of which lasted at least 10 weeks [14]. Children and students had to rely more on their resources to continue learning remotely via the Internet and other media, despite concerted efforts by the education community to ensure continuity of learning during this time [15]. Teachers also had to adapt to new pedagogical approaches and teaching methods for which they had not been adequately prepared [16,17].

During the COVID-19 pandemic, e-learning supported the abilities of students, academic staff, and professionals to develop professionally and personally [18,19,20]. Most higher education institutions offered online courses to students both on and off campus during the pandemic, and some have continued this practice [21,22]. Government spending on “education” as a percentage of GDP at the EU level has fluctuated between 4.7% and 5.1% between 1995 and 2021. In 1995, it was 4.9% of GDP, stagnated over the next decade, and declined to 4.7% in 2007. During the COVID-19 pandemic, government spending on education as a percentage of GDP at the EU level increased to 5.0% by 2020 [23].

However, the adoption of e-learning in response to the isolation caused by COVID-19 was not universally positive. During the pandemic, students with sufficient resources who were motivated and able to learn found alternative learning options, even when their schools or universities were not open [24]. Those from resource-poor backgrounds were often excluded when their schools or universities were closed [25,26]. Therefore, this crisis exposed the many deficiencies and inequities in education systems worldwide, from the lack of broadband access to the availability of portable devices [27,28]. Nevertheless, various strategies to bridge the digital divide in e-learning were implemented during the pandemic [29].

During the pandemic, several advantages and disadvantages of e-learning became apparent. Among the positive aspects, virtual classrooms allowed instructors to communicate effortlessly with students about their assignments and homework [30,31,32]. The negative aspects included that students could passively attend and leave lectures whenever they wanted and they were provided with little or no feedback from the instructor [30,31]. In addition, isolation during the pandemic contributed to psychological problems such as lower concentration, loneliness, and sleep problems. For this reason, during and after the pandemic, higher education institutions shifted their focus from a purely virtual to a hybrid approach by combining e-learning and a “talk and talk” approach [33,34].

Future educational programs must address the sense of community that was missing during the pandemic as it was dominated by online learning [35,36]. For this reason, many universities in the post-pandemic period are embracing hybrid degree programs, which offer a better solution, to ensure that students gain hands-on experience and have face-to-face contact so that they can stay motivated and take advantage of on-campus facilities and support while enjoying the flexibility of distance learning.

Although the COVID-19 pandemic has almost disappeared from European countries due to vaccinations, its effects have exposed the vulnerability of countries in their ability to adapt to crises. Moreover, new pandemics are likely to increase due to increased traffic and populations [37]. Therefore, it is of utmost importance to study the variations among countries in e-learning adoption during the most recent pandemic, along with the impact of economic development and the severity of the pandemic response on the use of e-learning.

3. Methodology

This article compares the adoption of e-learning in European countries before and during the pandemic. To examine the relationship between e-learning adoption, economic development, and the severity of the pandemic response, we use fuzzy C-means clustering, ANOVA, and a post hoc analysis. In this section, we present the data sources and the methodological approach.

3.1. Data Sources

This study focuses on the impact of the COVID-19 pandemic on the adoption of e-learning in European countries. For this purpose, we compared two periods: before and during the pandemic. The outbreak of COVID-19 occurred in 2019 and 2020 can be considered a year that was fully affected by the pandemic. Because changes in technology use during a particular year could be due to methodological errors or random fluctuations, we did not choose 2018 as the base year. Instead, we opted for two years prior to the pandemic and thus used 2017 as the base year (prior to COVID-19) and 2020 as the observation year (during COVID-19).

We used three data sources. First, we collected data from the Eurostat database on e-learning (e.g., the percentage of people taking an online course). Second, we collected data from the Oxford COVID-19 Government Response Tracker (OxCGRT) (e.g., the Average Stringency Index) as an indicator of the severity of the response to the COVID-19 outbreak. Third, we used the Gross Domestic Product (GDP) per capita, which is a good indicator of economic development. The severity indices [38] were taken from the Oxford COVID-19 Government Response Tracker (OxCGRT). The severity index (Avg_string_index) represents the average measure based on 9 response indicators (closure of schools, closure of workplaces, cancellation of public events, restrictions on meetings, closure of public transport, requests to stay at home, restrictions on domestic movement, international travel restrictions, and public information campaigns). In contrast, the strict reopening index (SI_reopen) uses the same 9 response indicators but measures the days a country was fully open. The research variables are explained in

Table 1. Research variables.

Variable Code	Measurement; Year	Description
E-learning indicators
ONLINE_C_2017	% of individuals; 2017	Doing an online course (any subject)
ONLINE_C_2020	% of individuals; 2020
ONLINE_M_2017	% of individuals; 2017	Using online learning materials
ONLINE_M_2020	% of individuals; 2020
ONLINE_COM_2017	% of individuals; 2017	Communicating with instructors or students using educational websites/portals
ONLINE_COM_2020	% of individuals; 2020
ONLINE_ANY_2017	% of individuals; 2017	Doing any of the e-learning activities
ONLINE_ANY_2020	% of individuals; 2020
ONLINE_C_M_2017	% of individuals; 2017	Doing an online course (any subject) or using online learning materials
ONLINE_C_M_2020	% of individuals; 2020
Economic indicators
GDP_PC_2017	Absolute value; 2017	GDP per capita in EUR
GDP_PC_2020	Absolute value; 2020
COVID-19 indicators
Avg_string_index	Measure 0–100, with 100 being the strictest response; 2020	The Average Stringency Index is a composite measure based on 9 response indicators (school closures, workplace closures, public event cancellations, meeting restrictions, public transportation closures, requests to stay home, restrictions on internal movement, international travel restrictions, and public information campaigns)
SI_reopen	Measure 0–100 with 100 being completely open; 2020	Stringency index at the reopening of a country; shows days of reopening. The values of the SI_reopening index range from 0 to 100 and measure the days when there were no reported COVID-19 cases.

Source: Authors’ work based on Eurostat data and the Oxford COVID-19 Government Response Tracker.

for both 2017 and 2020.

3.2. Methodological Process

Table 2. Detailed methodological framework.

	Step 1	Step 2	Step 3
Research Question	RQ1	RQ2	RQ3
Presumption	COVID-19 had a significant impact on the e-learning indicators	The European countries can be divided into homogeneous groups based on the e-learning indicators in the periods before and during the pandemic	COVID-19 influenced the digitization of countries that had lower economic development in the period before COVID-19 and thus had a significant impact on e-learning
Observed variables	e-learning indicators in 2017 and 2020	e-learning indicators in 2017 and 2020	Cluster membership; economic and COVID-19 indicators in the period before and during COVID-19 across clusters
Data sources	Eurostat	Eurostat	Results of Step 2; Oxford COVID-19 Government Response Tracker
Statistical methods	Descriptive statistics; box plot; paired t-test	Fuzzy C-means clustering, ANOVA	Descriptive statistics (economic and COVID-19 indicators); ANOVA; post hoc LSD test
Expected results of the analysis	Significant differences between e-learning in 2020 and 2017	Significant differences between clusters in e-learning indicators in 2020 and 2017	Significant differences between GDP per capita and COVID-19 indicators between clusters
Expected conclusion	European countries have responded appropriately by increasing e-learning	The level of e-learning in European countries is not uniform, with different implementation levels of e-learning in the periods before and during the pandemic	Both in the period before COVID-19 and the period during COVID-19, the more developed countries used e-learning to a greater extent

Source: Authors’ work.

shows the detailed methodological framework, which consisted of three phases. The table includes the following elements of the research questions defined in the introduction: the conjecture of the research questions, observed variables, data sources used, statistical methods, expected results of the analysis, and expected conclusions.

For example, the second research question (RQ2) examines whether European countries implemented e-learning uniformly in the periods before and during the pandemic.

It was assumed that European countries could be divided into homogeneous groups based on the e-learning indicators in the periods before and during the pandemic.

The e-learning indicators from Eurostat for 2017 and 2020 were used and analyzed using fuzzy C-means clustering and ANOVA.

We assumed that the differences in the e-learning indicators between the 2020 and 2017 clusters indicated that the level of e-learning was not uniform across European countries, with different levels of implementation of e-learning in the periods before and during the pandemic.

Figure 1 shows a summary of the methodological framework, which consisted of three steps.

First, in Step 1, the descriptive statistics of the e-learning indicators were calculated, comparing the e-learning indicators in 2017 and 2020 using box plots and t-paired tests.

Second, the fuzzy C-means clustering analysis identified the research clusters for 2017 and 2020. In addition, ANOVA was performed for the e-learning indicators in 2017 and 2020 and a comparison is shown on a country map.

Third, we compared the cluster members using stringency indicators representative of the severity of the pandemic response and GDP per capita using descriptive statistics, ANOVA, and a post hoc LSD test.

3.2.1. Step 1: Analysis of E-Learning in 2017 and 2020 in Selected European Countries

We compared the level of e-learning in European countries using a descriptive analysis for 2017 and 2020, including all EU countries, as well as Iceland, Kosovo, Montenegro, Northern Macedonia, Switzerland, Turkey, Serbia, and the United Kingdom. The descriptive analysis, box plots, and t-paired tests were used to compare the e-learning variables for 2017 and 2020.

3.2.2. Step 2: Fuzzy C-Means Clustering Analysis of E-Learning Indicators in 2017 and 2020

Cluster analysis is widely used to study the digital divide and national differences in technology use [39]. In this paper, we used fuzzy C-means clustering analysis, a technique developed by Jim Bezdek [40] in 1973 and improved and extended in 1981 [41,42]. The main objective of this technique is to overcome the uncertainty and imprecision of data sources and research hypotheses. Unlike other methods such as k-means, fuzzy C-means clustering allows the researcher to represent data points in different clusters based on probability [43]. The fuzzy C-means clustering algorithm uses fuzzy partitioning so that the data points can belong to each group with different degrees of 0 and 1. It is also suitable for the case of unit variability, making it a good solution for studying the digital divide [44,45,46].

The fuzzy C-means clustering algorithm has been widely used for e-learning analysis [47,48,49]. As suggested by [42], this algorithm was selected as a suitable method for research due to its ability to generate approximate solutions in a more expedient manner, as well as its ability to address problems related to understanding patterns, incomplete/noisy data, mixed-media information, and human interaction.

Fuzzy C-means clustering was used to identify clusters of European countries according to the e-learning indicators for 2017 and 2020. The following steps used in the fuzzy C-means clustering analysis were based on [50].

Firstly, a value for 𝑐 was fixed, where 𝑐 is (2 < 𝑐 < 𝑛), followed by selecting the value for parameter 𝑚′ to initialize the partition matrix, 𝑈-(0). Each step was labeled as 𝑟, where 𝑟 = 0, 1, 2…𝑡. In the next phase, the center vector {,-𝑖.} for each step 𝑐 was calculated, as in Equation (1):

$$V_{ij}=\frac{{{\displaystyle \sum }}_{k=1}^{n}u\frac{m^{\prime }}{ik} x x_{kj} }{{{\displaystyle \sum }}_{k=1}^{n}u\frac{m^{\prime }}{ik}}$$

(1)

where

$V_{ij}$ is the degree of membership of $V_i$ in the cluster j,

$x_i$ is the ith of the d-dimensional measured data,

$c_j$ is the d-dimensional center of the cluster.

In the next phase, the distance matrix $D_{\left[c,n\right]}$ was calculated:

$$D_{ij}=[{\displaystyle {\displaystyle \sum }_{j=1}^m}{[x_{kj}-v_{ij}]}^2{]}^{\left[\frac{1}{2}\right]}$$

(2)

In the final phase, the partition matrix needed to be updated for the $r^{th}$ step, $U^{\theta },$ as follows:

$$\begin{array}{c}u\begin{array}{c}r-1\\ ik\end{array}=1\\ {\displaystyle {\displaystyle \sum }_{j=1}^c}[\frac{d_{ik}^r}{d_{jk}^r}{]}^{2/\left[m^{\prime }-1\right]}\end{array}$$

(3)

if $\parallel U^{\left(k+1\right)}-U^{\left(k\right)}\parallel <\delta \text{then} \text{STOP},$ otherwise, return to STEP 2 by updating the cluster centers and grades for the data point.

Source: Authors’ work based on [45].

JASP ver. 0.17.1, the statistical software for fuzzy C-means clustering, was used to perform the cluster analysis and investigate the heterogeneity of European countries in terms of e-learning. The variables that characterize the level of e-learning described in Table 1 were used for clustering. The appropriate number of clusters was determined using the elbow technique and the homogeneity of the clusters was assessed using a t-Distributed Stochastic Neighbor Embedding (t-SNE) cluster plot. The following training settings were used: maximum number of iterations (25), fuzziness parameter (2), and optimization of the number of clusters (Silhouette optimization).

3.2.3. Step 3: The Economic and COVID-19 Indicators in E-Learning Clusters in 2017 and 2020

In the third step, in addition to the results from step 2, the economic and COVID-19 severity indicators were considered as proxies for the severity of the response to the pandemic. First, the GDP per capita in 2017 and 2020 were compared between clusters using ANOVA and a post hoc analysis. Second, the COVID-19 severity indices (the average severity index and the severity index at the reopening of the country) were compared between clusters in 2020 using ANOVA and a post hoc analysis. A post hoc LSD test was used to detect specific differences between individual clusters [51].

4. Results

This section presents the research results of the three steps described above.

4.1. Step 1: Analysis of E-Learning in 2017 and 2020 in Selected European Countries

In the first step, we analyzed the e-learning indicators in 2017 and 2020 in selected European countries to address the first research question (RQ1).

Table 3. Descriptive statistics for e-learning indicators in 2017 and 2020.

	Valid	Mean	Std. Dev.	Min.	Max.
ONLINE_C_2017	34	7.235	4.626	2.000	20.000
ONLINE_M_2017	34	16.441	11.789	3.000	71.000
ONLINE_COM_2017	34	9.529	5.316	2.000	24.000
ONLINE_ANY_2017	34	21.471	12.776	4.000	74.000
ONLINE_C_M_2017	34	18.853	12.263	3.000	73.000
ONLINE_C_2020	34	14.882	6.650	3.000	32.000
ONLINE_M_2020	34	21.824	11.761	6.000	72.000
ONLINE_COM_2020	34	16.088	6.995	3.000	30.000
ONLINE_ANY_2020	34	29.324	12.829	10.000	76.000
ONLINE_C_M_2020	34	26.059	12.170	8.000	75.000

Source: Authors’ work based on Eurostat data.

shows the descriptive statistics for the e-learning indicators in 2017 and 2020.

All e-learning indicators showed a strong increase in 2020 compared to 2017. The highest increase was in the variable representing people taking an online course (any subject) or using online learning materials (ONLINE_C), which increased by 7.7%. Figure 2 shows the mean values of the e-learning indicators in 2017 and 2020.

Table 4. Paired samples t-test of e-learning indicators in 2020 compared to 2017.

2017 vs. 2020	t	df	p	Mean Difference	SE Difference	Cohen’s d
ONLINE_C	11.412	33	<0.001 ***	7.647	0.670	1.957
ONLINE_M	5.035	33	<0.001 ***	5.382	1.069	0.864
ONLINE_COM	8.410	33	<0.001 ***	6.559	0.780	1.442
ONLINE_ANY	8.472	33	<0.001 ***	3.265	0.385	1.453
ONLINE_C_M	6.969	33	<0.001 ***	7.206	1.034	1.195

Note: *** statistically significant at 1%. Source: Authors’ work based on Eurostat data.

shows the paired samples t-test. The differences in the e-learning indicators between 2020 and 2017 were significantly below the 1% level.

The largest difference (7.647) was observed in the variable measuring the percentage of people taking an online course (any subject) (ONLINE_C). In contrast, the smallest difference (3.265) was observed in the variable measuring the percentage of individuals who used online learning materials (ONLINE_M). This result probably indicates that during the pandemic, the use of e-learning increased the most in formal education (measured by participation in an online course). Informal education (measured by the use of online learning materials), on the other hand, fell behind.

These results were confirmed by Cohen’s d, which relates the difference in means to the variability. Cohen’s d is calculated as the ratio between the mean differences and the pooled standard deviation. A higher Cohen’s d indicates that the mean difference is greater than the variability measured by the pooled standard deviation. The variable measuring the percentage of people taking an online course (any subject) (ONLINE_C) had the highest Cohen’s d value (1.957). In contrast, the lowest value (0.864) was shown for the variable measuring the percentage of individuals who used online learning materials (ONLINE_M).

Figure 3 shows the box plots for the e-learning indicators in 2017 and 2020. It can be seen that for the majority of variables, the country listed at number 14 (Iceland) was the biggest outlier, followed by the countries numbered 7 (Denmark) and 9 (Finland) in 2020 and 31 (Switzerland) in 2017.

4.2. Step 2: Fuzzy C-Means Clustering Analysis of E-Learning Indicators for 2017 and 2020

In the second step, we used the fuzzy C-means clustering analysis to divide the European countries into homogeneous groups. The validity of the cluster was determined using similarity measures that evaluated intensity, connectivity, and distance. The elbow method was used to determine the number of clusters. In addition, the validity of the cluster solution was demonstrated using a t-SNE map by visualizing the map in a high-dimensional space. ANOVA was used to confirm the relevance of the e-learning variables in the cluster analysis.

4.2.1. Determining the Number of Clusters

Figure 4 shows the plot of the elbow method. This method is used to determine the number of clusters using indicators such as the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and Within-Cluster Sum of Squares (WSS). The AIC and BIC are measures of model performance that are used to determine the complexity of a model and WSS measures the distance of each cluster observation from the cluster center [52]. The elbow plot shows (on the y-axis) the AIC, BIC, and WCSS values for different cluster numbers (on the x-axis). When an elbow shape is observed in a graph, the number of clusters at which the elbow is formed is selected. This location is called the elbow point. Increasing the number of clusters beyond the elbow point does not significantly decrease the AIC, BIC, or WSS. The first elbow can be seen at three, which means that the solution with three clusters is optimal for the 2017 and 2020 datasets.

4.2.2. Cluster Validation

t-SNE is a method used to plot high-dimensional data, where each point is assigned a position on a two- or three-dimensional graph. The approach is a variant of Stochastic Neighbor Embedding (SNE), which is much easier to optimize and produces much better visualizations [53]. t-SNE can capture a significant portion of the local structure of high-dimensional data while displaying the global structure, such as clusters at many scales, using an unsupervised machine learning algorithm. It shows the probability distribution of similar data points, which are later transferred into clusters. Figure 5 shows the t-SNE cluster plot for the 2017 and 2020 cluster solutions. Since the cluster members shown in different colors are close to each other, the plot shows that the cluster solutions are valid.

The cluster solution with three clusters had the optimal AIC and BIC values, as shown in Figure 4.

Table 5. AIC, BIC, and Silhouette indicators of the fuzzy C-means clustering solution for 2017 and 2020.

Year	Clusters	N	R²	AIC	BIC	Silhouette
2017	3	34	0.570	97.510	120.410	0.260
2020	3	34	0.466	94.460	117.360	0.320

Source: Authors’ work based on Eurostat data.

shows the AIC, BIC, and Silhouette indicators of the fuzzy C-Means clustering solution for 2017 and 2020, which measured whether the data were interpreted consistently within the clusters [54]. The Silhouette score ranged from −1 to 1 and should be maximized to obtain the optimal clustering solution, whereas the BIC and AIC values should be minimized. All three indicators performed better for the solution calculated for 2020 than for the one calculated for 2017.

Table 6. Evaluation metrics of the fuzzy C-means clustering solution for 2017 and 2020.

Metrics	2017	2020
Pearson’s γ	0.393	0.388
Calinski–Harabasz index	22.383	24.173
Dunn index	0.030	0.051
Entropy	1.003	1.058

Note: All metrics are based on the Euclidean distance. Source: Authors’ work based on Eurostat data.

shows additional evaluation metrics for the cluster solutions for 2017 and 2020: Pearson’s γ, the Calinski–Harabasz index, the Dunn index, and the entropy.

Pearson’s γ-measure determines the Pearson correlation for all pairwise distances against a vector of ones and zeros, where 0 indicates that they are in the same cluster and 1 indicates that they are in different clusters [55]. According to the Calinski–Harabasz index, successful clustering has the highest proportion of variation that the clusters can explain [56]. This measure compares the sum of squares between and within clusters. The Dunn index is based on the dissimilarity of clusters and the maximum diameter [57]. Finally, a highly ordered system, such as a dense cluster solution, has very low entropy and a highly disordered system has high entropy [58]. Based on all the evaluation metrics, except for the entropy, the density of the clusters increased in 2020 compared to 2017.

Table 7. ANOVA results of the differences in e-learning for the cluster solutions (2017 and 2020).

		Sum of Squares	df	Mean Square	F	Sig.
2017
ONLINE_C_2017	Between Groups	477.940	2	238.970	32.466	<0.001 ***
	Within Groups	228.178	31	7.361
	Total	706.118	33
ONLINE_M_2017	Between Groups	2045.171	2	1022.586	12.474	<0.001 ***
	Within Groups	2541.211	31	81.975
	Total	4586.382	33
ONLINE_COM_2017	Between Groups	681.626	2	340.813	42.119	<0.001 ***
	Within Groups	250.844	31	8092
	Total	932.470	33
ONLINE_ANY_2017	Between Groups	3110.426	2	1555.213	21.182	<0.001 ***
	Within Groups	2276.044	31	73.421
	Total	5386.470	33
ONLINE_C_M_2017	Between Groups	2595.320	2	1297.660	16.996	<0.001 ***
	Within Groups	2366.944	31	76.353
	Total	4962.264	33
		Sum of Squares	df	Mean Square	F	Sig.
2020
ONLINE_C_2020	Between Groups	940.101	2	470.050	28.053	<0.001 ***
	Within Groups	519.429	31	16.756
	Total	1459.529	33
ONLINE_M_2020	Between Groups	1890.062	2	945.031	10.952	<0.001 ***
	Within Groups	2674.879	31	86.286
	Total	4564.941	33
ONLINE_COM_2020	Between Groups	1254.444	2	627.222	53.967	<0.001 ***
	Within Groups	360.291	31	11.622
	Total	1614.735	33
ONLINE_ANY_2020	Between Groups	3597.507	2	1798.754	30.405	<0.001 ***
	Within Groups	1833.934	31	59.159
	Total	5431.441	33
ONLINE_C_M_2020	Between Groups	2684.185	2	1342.092	18.880	<0.001 ***
	Within Groups	2203.698	31	71.087
	Total	4887.882	33

Note: *** statistically significant at 1%. Source: Authors’ work based on Eurostat data.

shows the ANOVA results for the e-learning indicators for 2017 and 2020. The ANOVA compared the variances between the means of the different e-learning variables used in the study. The significance value indicates whether the clusters differed from each other with respect to the respective e-learning measure.

In this study, all variables were found to have statistical significance below the 1% level, indicating that the differences between clusters were statistically significant for both 2017 and 2020.

4.2.3. Cluster Solution

Table 8. Fuzzy C-means clustering solutions for 2017 and 2020.

	2017			2020
Cluster	Cluster 1	Cluster 2	Cluster 3	Cluster 1	Cluster 2	Cluster 3
Size	10	18	6	14	13	7
Explained proportion within-cluster heterogeneity	0.783	0.196	0.021	0.126	0.809	0.065
Within-sum of squares	52.877	13.248	1.385	8.131	52.164	4.170

Source: Authors’ work based on Eurostat data.

shows the fuzzy C-means clustering solutions for 2017 and 2020. The table contains information about the cluster size, cluster heterogeneity, and corresponding sum of squares.

The results show that Cluster 1 in 2017 and Cluster 2 in 2020 had the highest explained proportions of within-cluster heterogeneity and within-sum of squares. On the other hand, in both 2017 and 2020, Cluster 3 had the lowest explained proportion of within-cluster heterogeneity and within-sum of squares.

Table 9. E-learning indicators according to clusters in 2017 and 2020.

Cluster	Descriptives	ONLINE_C	ONLINE_M	ONLINE_COM	ONLINE_ANY	ONLINE_C_M
2017
Cluster 1	Mean	12.90	27.70	16.10	35.20	31.50
N = 10	Std. Dev.	4.254	15.791	3.446	14.382	15.255
	Var. Coeff.	0.33	0.57	0.21	0.41	0.48
Cluster 2	Mean	5.44	13.61	7.77	18.22	15.72
N = 18	Std. Dev.	1.822	3.928	2.819	4.735	3.770
	Var. Coeff.	0.33	0.28	0.36	0.26	0.24
Cluster 3	Mean	3.16	6.16	3.83	8.33	7.16
N = 6	Std. Dev.	1.329	2.639	1.329	2.582	2.483
	Var. Coeff.	0.42	0.43	0.35	0.31	0.35
Total	Mean	7.23	16.44	9.52	21.47	18.85
N = 34	Std. Dev.	4.626	11.789	5.316	12.776	12.263
	Var. Coeff.	0.64	0.71	0.55	0.59	0.65
2020
Cluster 1	Mean	21.00	30.23	23.38	41.38	36.31
N = 13	Std. Dev.	5.339	14.388	3.906	11.362	12.822
	Var. Coeff.	0.25	0.47	0.17	0.27	0.35
Cluster 2	Mean	13.00	19.71	13.36	25.43	23.07
N = 14	Std. Dev.	3.038	3.099	2.872	3.936	3.540
	Var. Coeff.	0.23	0.15	0.21	0.15	0.15
Cluster 3	Mean	7.29	10.43	8.00	14.71	13.00
N = 7	Std. Dev.	3.094	3.309	3.416	3.729	3.367
	Var. Coeff.	0.42	0.32	0.43	0.25	0.26
Total	Mean	14.88	21.82	16.09	29.32	26.06
N = 34	Std. Dev.	6.650	11.761	6.995	12.829	12.170
	Var. Coeff.	0.45	0.54	0.44	0.44	0.47

Source: Authors’ work based on Eurostat data.

shows the mean values of the e-learning indicators for the clusters in 2017 and 2020, which are also shown in Figure 6. The mean values of all the variables increased for all clusters from 2017 to 2020. The best-performing e-learning indicator for each cluster was the variable measuring the percentage of people doing any e-learning activities (ONLINE_ANY) with the highest mean value overall. The mean values of all variables measuring e-learning behavior were the highest for Cluster 1 and the lowest for Cluster 3.

4.2.4. Cluster Members and Characteristics

Table 10. Countries by cluster in 2017 and 2020.

2017		2020
Cluster	Countries	Cluster	Countries
Cluster 1 (n = 10)	Estonia, Finland, Iceland, Luxembourg, Malta, Netherlands, Norway, Spain, Sweden, the United Kingdom	Cluster 1 (n = 14)	Belgium, Denmark, Estonia, Finland, Iceland, Luxembourg, Malta, Netherlands, Norway, Portugal, Spain, Sweden, Switzerland
Cluster 2 (n = 18)	Austria, Belgium, Cyprus, Czechia, Denmark, France, Germany, Ireland, Italy, Latvia, Lithuania, Montenegro, Portugal, Romania, Serbia, Slovakia, Slovenia, Switzerland	Cluster 2 (n = 13)	Austria, Croatia, Cyprus, France, Germany, Hungary, Ireland, Italy, Latvia, Lithuania, Montenegro, Slovakia, Slovenia, the United Kingdom
Cluster 3 (n = 6)	Bulgaria, Croatia, Greece, Hungary, Poland, Turkey	Cluster 3 (n = 7)	Bulgaria, Czechia, Greece, Poland, Romania, Serbia, Turkey

Source: Authors’ work based on Eurostat data.

lists the countries by cluster. There was an increase in the number of countries in Cluster 1 (from 10 to 14) and Cluster 3 (from 6 to 7), whereas there was a decrease in the number of countries in Cluster 2 (from 18 to 13). There was also a change in the participating countries in each cluster.

Several countries switched from Cluster 2 in 2017 to Cluster 1 in 2020 (Belgium, Denmark, Portugal, Switzerland). In addition, only the United Kingdom moved from Cluster 1 in 2017 to Cluster 2 in 2020, whereas Croatia and Hungary moved from Cluster 3 in 2017 to Cluster 2 in 2020, indicating a moderately developed European country. Finally, Cluster 3 represented the lowest level of e-learning among the European countries in the sample and the Czech Republic, Romania, and Serbia moved from Cluster 2 in 2017 to Cluster 3 in 2020.

Figure 7 shows a map of European countries. Red represents Cluster 1, blue represents Cluster 2, and green represents Cluster 3.

In summary, the results of this section provide information that addresses the second research question (RQ2), which examines the extent of the homogeneity of e-learning in European countries during the pandemic. The research results show that the cluster solution generated by fuzzy C-means clustering for 2017 and 2020 had a high degree of homogeneity and that the countries in the periods both before and during the pandemic can be grouped into meaningful clusters.

4.3. Step 3: The Impact of Economic and COVID-19 Indicators on E-Learning Clusters for 2017 and 2020

Table 11. Descriptive statistics for the economic indicator (Gross Domestic Product per capita in EUR) in 2017 and 2020 according to clusters.

	GDP_PC_2017			GDP_PC_2020
	Cluster 1	Cluster 2	Cluster 3	Cluster 1	Cluster 2	Cluster 3
Valid	18	10	6	14	13	7
Mean	33,360.000	24,964.000	15,793.333	29,102.214	30,586.154	19,812.857
Std. Dev.	12,506.964	14,989.428	10,669.414	13,037.195	18,187.007	11,314.240
Shapiro-Wilk	0.952 **	0.933 **	0.781 **	0.877 **	0.928 **	0.863 **

Note: ** statistically significant at 5%. Source: Authors’ work based on Eurostat data.

shows the descriptive statistics for the GDP per capita for 2017 and 2020.

Cluster 1 had the highest mean GDP per capita in 2017, whereas Cluster 2 had the highest mean GDP per capita in 2020.

Cluster 3 consistently had the lowest GDP per capita in both 2017 and 2020. These results are consistent with countries shifting from one cluster to another, such as the United Kingdom’s shift from Cluster 1 to Cluster 2 from 2017 to 2020.

The results of the Shapiro–Wilk test show that the GDP per capita was normally distributed across all clusters in both of the observed years, allowing for the use of ANOVA to test the significance of the differences between clusters.

The ANOVA results (

Table 12. ANOVA results according to clusters for economic indicators and COVID-19 indices.

		Sum of Squares	df	Mean Square	F	Sig.
GDP_PC_2017	Between Groups	1,501,831,014.902	2	750,915,507.451	4.434	0.020 **
	Within Groups	5,250,538,973.333	31	169,372,224.946
	Total	6,752,369,988.235	33
GDP_PC_2020	Between Groups	571,153,709.829	2	285,576,854.914	1.274	0.294
	Within Groups	6,946,868,602.907	31	224,092,535.578
	Total	7,518,022,312.735	33

Note: ** statistically significant at 5%. Source: Authors’ work based on Eurostat data.

) show that the differences in GDP per capita among cluster members in 2017 are statistically significant at the 5% level. However, in 2020, the differences in GDP per capita among cluster members are no longer statistically significant.

Table 13. Descriptive statistics for average stringency index and average stringency index in days of reopening in 2020 according to clusters.

	Avg_Stringency_Index_2020			SI_Days_Reopen
	Cluster 1	Cluster 2	Cluster 3	Cluster 1	Cluster 2	Cluster 3
Valid	13	13	7	13	12	7
Missing	0	1 *	0	1 *	1 *	0
Mean	51.869	58.194	57.139	85.470	73.072	83.996
Std. Dev.	7.273	6.492	5.236	8.623	12.965	8.413
Shapiro-Wilk	0.976	0.923	0.956	0.923	0.870	0.923

Note: * Data is missing for Montenegro. Source: Authors’ work based on Eurostat data.

shows the descriptive statistics for the COVID-19 severity indicators used as proxies for the severity of the pandemic response for the clusters in 2020.

The COVID-19 indicators are represented by the average severity index (Avg_stringency_index) and the severity index for reopening the country (SI_days_reopen).

One might expect that stringency, as an indicator of the severity of the response to the pandemic, would be the highest in countries that implemented the most e-learning during the COVID-19 pandemic, as the countries most affected by the pandemic had a greater need for e-learning as a measure to overcome isolation. However, the results are counterintuitive.

Countries in Cluster 1, where e-learning was most prevalent, had the lowest average stringency index (51.869) and the highest stringency index for reopening the country (85.470). These results are actually the opposite of what one might expect given that the lower average stringency index and the higher stringency index for reopening the country indicate a less severe impact of the pandemic. Considering that the countries from Cluster 1 are the most economically developed, they could still push for the adoption of e-learning even if they were not the most affected by the pandemic.

Table 14. ANOVA analysis according to clusters of economic indicators and COVID-19 indices.

		Sum of Squares	df	Mean Square	F	Sig.
GDP_PC_2017	Between Groups	1,501,831,014.902	2	750,915,507.451	4.434	0.020 **
	Within Groups	5,250,538,973.333	31	169,372,224.946
	Total	6,752,369,988.235	33
GDP_PC_2020	Between Groups	571,153,709.829	2	285,576,854.914	1.274	0.294
	Within Groups	6,946,868,602.907	31	224,092,535.578
	Total	7,518,022,312.735	33
Avg_stringency_indeks_2020	Between Groups	284.521	2	142.261	3.271	0.052 *
	Within Groups	1304.882	30	43.496
	Total	1589.403	32
SI_reopen	Between Groups	1068.813	2	534.407	4.895	0.015 **
	Within Groups	3166.126	29	109.177
	Total	4234.939	31

Note: ** statistically significant at 5%; * 10%. Source: Authors’ work based on Eurostat data.

shows the ANOVA results for the differences between the average severity index (Avg_stringency_index) and the severity index for reopening the country (SI_days_reopen). The results indicate that there are significant differences between cluster members in 2020 at the 10% level for the average stringency index and at the 5% level for the severity index for reopening the country.

Although the ANOVA provided information about whether there were significant differences between the clusters, it did not provide accurate information about specific clusters that were significantly different. For this purpose, we used the Least Significant Difference (LSD) post hoc test (

Table 15. Post hoc LSD test for differences between clusters according to the economic indicator and stringency indices.

Dependent Variable	Cluster Group		Mean Difference (I-J)	Std. Error	Sig.
GDP_PC_2017	Cluster 1	Cluster 2	−8396.000	5132.912	0.112
		Cluster 3	9170.667	6720.560	0.182
	Cluster 2	Cluster 1	8396.000	5132.912	0.112
		Cluster 3	17,566.667	6135.004	0.007 ***
	Cluster 3	Cluster 1	−9170.667	6720.560	0.182
		Cluster 2	−17,566.667	6135.004	0.007 ***
Avg_stringency_indeks_2020	Cluster 1	Cluster 2	6.324	2.586	0.021 **
		Cluster 3	1.055	3.091	0.735
	Cluster 2	Cluster 1	−6.324	2.586	0.021 **
		Cluster 3	−5.269	3.091	0.099
	Cluster 3	Cluster 1	−1.055	3.091	0.735
		Cluster 2	5.269	3.091	0.099
SI_reopen	Cluster 1	Cluster 2	12.398	4.182	0.006 ***
		Cluster 3	1.474	4.898	0.766
	Cluster 2	Cluster 1	−12.398	4.182	0.006 ***
		Cluster 3	−10.924	4.969	0.036 **
	Cluster 3	Cluster 1	−1.474	4.898	0.766
		Cluster 2	10.924	4.969	0.036 **

Note: *** statistically significant at 1%; ** 5%. Source: Authors’ work based on Eurostat.

). The results showed that the economic indicator (GDP per capita in EUR) influenced the relationship between Cluster 2 and Cluster 3 by 1% in 2017. Moreover, COVID-19 indicators, such as the average stringency index in 2020, had a significant relationship between Cluster 1 and Cluster 2 of below 5%. On the other hand, the stringency reopening index showed a statistically significant relationship between Cluster 1 and Cluster 2 of below 1% and between Cluster 3 and Cluster 1 of below 5%.

The results presented provide insights into the third research question (RQ3), which examines the estimated impact of GDP per capita on e-learning before and during the COVID-19 pandemic. The results suggest that economic development had a significant impact on e-learning adoption in the pre-pandemic period while it was no longer relevant during the pandemic. In contrast, during the pandemic, stringency measures differed statistically across clusters. This indicates that the pandemic had a greater and more positive impact on the use of e-learning in the countries in the sample that were grouped into clusters based on economic prosperity. However, the effects were counterintuitive, suggesting that the countries with the less stringent pandemic measures were able to use e-learning to a greater extent, likely due to their higher levels of economic development.

5. Discussion and Conclusions

5.1. Summary of the Research

Due to the intensity of the COVID-19 outbreak, educational institutions, among others, experienced levels of social isolation never before experienced. On the positive side, digital technologies, such as cloud computing and video broadcasting, have accelerated the adoption of e-learning. However, the speed and effectiveness of e-learning adoption during the pandemic varied across countries.

This article examined the adoption of e-learning in European countries before and during the pandemic and the relationship between the pandemic, e-learning, and economic development. The research findings suggest that before and during the pandemic, economically and digitally developed nations led the way in e-learning adoption.

First, we compared the adoption of e-learning in European countries before and during the pandemic to address the first research question (RQ1). The overall average increase for the e-learning variables for the countries sampled in this research was 6.94%. The largest increase was observed for the variable ONLINE C, which represents those taking an online course (any topic) or using online learning materials, with an increase of 7.7%.

Second, fuzzy C-Means clustering was used to create clusters of European countries for two periods (before and during the pandemic) to investigate the homogeneity of European countries in adopting e-learning and address the second research question (RQ2). Using the elbow method in combination with different validity indicators, three clusters were extracted for the two periods, confirming the strong homogeneity among European countries in terms of e-learning adoption. The countries in Cluster 1 were mainly located in Northern Europe and were the most advanced in terms of economic development and e-learning adoption. The countries in Cluster 2 were mainly located in Central Europe and were the second best in terms of economic development and e-learning adoption. The countries in Cluster 3 were mainly located in Southern Europe and were behind in terms of economic development and e-learning adoption. Several countries moved out of the different clusters during the COVID-19 pandemic.

The degree of homogeneity was the highest for Cluster 3, which in 2017, included Bulgaria, Croatia, Greece, Hungary, Poland, and Turkey. The countries in Cluster 3 remained almost unchanged in 2020, with the Czech Republic, Romania, and Serbia joining from Cluster 1, whereas Croatia moved to Cluster 2. The second-highest homogeneity was found for Cluster 1, whose structure improved in 2020 compared to 2017. The countries belonging to Cluster 1 in 2020 were Austria, Croatia, Cyprus, France, Germany, Hungary, Ireland, Italy, Latvia, Lithuania, Montenegro, Slovakia, Slovenia, and the United Kingdom. Finally, Cluster 2 was found to have the lowest homogeneity.

Third, we compared the GDP per capita as an indicator of economic development and the severity indices as an indicator of the severity of the response to the pandemic in the different clusters to address the third research question (RQ3). The results indicate that the countries with the highest average GDP per capita both before and during the pandemic rapidly adopted e-learning, even though they had the least severe response to the pandemic. At the same time, less-developed countries had a more rigorous response to the pandemic due to their limited healthcare resources but were still behind in terms of e-learning adoption.

5.2. Main Conclusions

The main conclusions of this research are as follows.

First, the Mathews effect on e-learning adoption persisted during the pandemic [5]. Thus, the digital divide was evident both before and during the pandemic. The countries that were ahead in e-learning adoption before the pandemic were also ahead during the pandemic, and the countries that were behind in e-learning before the pandemic were also behind during the pandemic.

Second, there were some exceptions in e-learning adoption and cluster membership in terms of countries’ mobility from less advanced to more advanced clusters (Table 10). Several countries, such as Belgium, Denmark, Portugal, Switzerland, and Croatia, advanced their use of e-learning due to the pandemic. These countries moved up from the lower clusters to the higher clusters (e.g., from Cluster 3 to Cluster 2 in the case of Croatia). On the other hand, only a few countries moved from the more advanced clusters to the less advanced ones.

Third, the countries with the highest average GDP per capita both before and during the pandemic rapidly adopted e-learning, although they had the weakest response to the pandemic. However, the differences between clusters in terms of average GDP per capita were statistically different only before the pandemic. During the pandemic, on the other hand, the differences were no longer statistically significant, even though the most developed countries had the highest penetration of e-learning. Less-developed countries were more responsive to the pandemic due to their poor healthcare but they also fell behind in e-learning adoption.

5.3. Practical Implications

The following are practical recommendations for implementing the conclusions summarized above.

First, strong policy initiatives should be made at the EU level to address the close relationship between economic development and e-learning adoption. These policies should aim to use e-learning to better integrate students from geographically distant areas into educational institutions at all levels, as well as to better integrate students with disabilities. In addition to top-down initiatives, a bottom-up approach could also be used to develop resilient e-learning infrastructures at the country level.

Second, policymakers should examine the countries that rapidly improved their adoption of e-learning and moved from the less-advanced cluster to the more advanced cluster. This investigation could focus on micro and macro initiatives and develop a roadmap of best practices to improve e-learning adoption.

Third, although the DESI index currently covers a broad spectrum (human capital, connectivity, digital technology integration, and digital public services), it does not yet include indicators to measure e-learning. There have been proposals to complement the DESI index with a new measurement that reflects e-learning in all dimensions. For example, one of the e-learning indicators already covered by Eurostat (e.g., use of online courses) could be included in the human capital dimension.

Finally, this study could be of interest to software companies that specialize in e-learning, as their market grew strongly after the COVID-19 pandemic. Market-driven initiatives have been proven to be a valuable driver for bridging the digital divide, and their impact should be encouraged and further studied.

5.4. Research Limitations and Future Research Directions

The limitations of this study stem from the geographic and temporal coverage, as well as the research methodology.

First, the research was conducted on a sample of European countries. Therefore, examining a broad sample of countries would provide a more comprehensive perspective on the research questions. It would be interesting to compare data from countries outside Europe with the results of this research to verify that the GDP variable did not affect the adoption of e-learning in less-developed countries after the COVID-19 pandemic.

Second, the comparison of the periods before and during the pandemic only focused on two time periods. Examining a longer time series would provide insights into issues that were not covered by this study, e.g., in addition to examining the increase in e-learning, the year-to-year changes in the longer time series could also be compared, providing insights into the rate of change.