Online Learning from the Learning Cycle Perspective: Discovering Patterns in Recent Research

Abstract

We propose a method for automatically extracting new trends and best practices from the recent literature on online learning, aligned with the learning cycle perspective. Using titles and abstracts of research articles published in high ranked educational journals, we assign topic proportions to the articles, where the topics are aligned with the components of the learning cycle: engagement, exploration, explanation, elaboration, evaluation, and evolution. The topic analysis is conducted using keyword-based Latent Dirichlet allocation, and the topic keywords are chosen to reflect the nature of the learning cycle components. Our analysis reveals the time dynamics of research topics aligned on learning cycle components, component weights, and interconnections between them in the current research focus. Connections between the topics and user-defined learning elements are discovered. Concretely, we examine how effective learning elements such as virtual reality, multimedia, gamification, and problem-based learning are related to the learning cycle components in the literature. In this way, any innovative learning strategy or learning element can be placed in the landscape of the learning cycle topics. The analysis can be helpful to other researches when designing effective learning activities that address particular components of the learning cycle.

1. Background

Online learning applications with a large proportion of interactive, self-directed content for individual learning paths are becoming increasingly essential in higher education. The COVID-19 lockdown, in particular, spurred a boom in the use of such applications and in the empirical study of their effectiveness.

In their meta-analysis of more than 50 publications featuring randomized or quasi-randomized experiments on online, blended, and face-to-face learning, Means et al. [1] reported significant improvements in learning outcomes for online and blended formats. More recent meta-studies also suggest the superiority of the flipped classroom approach over traditional learning. Numerous studies have consistently shown that active online learning units within a flipped classroom model outperform traditional lecturing (Ying and Thompson [2], Shi et al. [3], Hew et al. [4]). Research on MOOCs and small online courses (e.g., Li et al. [5] and Zheng et al. [6]) further supports that fully asynchronous online learning can be effective, provided the course activities and materials are well designed to meet students’ needs.

This raises the questions of how to design an effective online learning unit and which learning elements enhance its efficiency. At the same time, research on enhancing these applications with innovative and effective learning features is rapidly evolving. Keeping up with the vast number of research articles that test and adopt new learning technologies—while continuously generating new insights—is challenging.

The purpose of this paper is to propose an approach to automatically discover patterns for designing online learning units using machine learning. Our primary goals are to structure recent relevant literature within a learning cycle framework, establish associations between the discovered topics and key elements of online learning units that contribute to learning success, and provide recommendations for designing effective online learning applications.

To conduct the analysis efficiently, we aim to use Latent Dirichlet Allocation (LDA, Blei et al. [7]) for automatic topic identification. As a commonly used technique for structuring text data automatically, topic analysis (Churchill and Singh [8]) represents each document in a collection (or corpus) as a combination of weighted keyword sets (topics), with each document containing these topics in varying proportions. In the context of e-learning literature, LDA was employed, e.g., by Gurcan et al. [9] to identify emerging trends. While suitable for tracking general trends and the time evolution of article volumes, their approach, which revealed broad topics such as “learning system” and “learning factors”, does not provide actionable recommendations for designing modern learning units in line with recent research findings.

To structure our analysis with a focus on designing learning applications, we adopted the 5E instructional framework, where the design of learning units follows the five-phase learning cycle: engage, explore, erxplain, elaborate, and evaluate, as described in Bybee et al. [10] and Hew et al. [4]. The first phase engages students by addressing a real-world problem or question, the second involves exploring the problem, and the third seeks to explain the phenomena using existing knowledge. The elaboration phase deepens understanding through additional material and exercises, while the final phase evaluates learning success.

Aligning the topics to the learning cycle components, as outlined above, allows for a structured representation of content that is helpful in designing effective learning applications. However, the LDA approach of Gurcan et al. [9] relies purely on data-driven topic extraction, offering no possibility for user-supplied input regarding topic content. Recently, Eshima et al. [11] proposed an LDA-based method called keyATM, which allows for the explicit embedding of known structures or side information into topic allocation. In this method, the authors embed user-supplied keywords into the LDA estimation procedure to obtain topics aligned with the user’s specific focus.

Thus, by using keyATM, we can extract topics aligned with the 5E learning cycle, combined with the evolution component, by tying the keywords to the definitions and descriptions of the stages discussed above. This approach enables us to analyze the interconnections between learning cycle stages, the dynamics of research volume devoted to each stage, and the relatedness of specific learning elements and tools to the learning cycle components.

The underlying text corpus consists of the titles and abstracts of articles published between 1 January 2019 and 31 March 2023. This time frame was chosen to build on the comprehensive review by Martin et al. [12], which covered works from 2009 to 2018 and identified 12 research themes in online learning. Our goal is to highlight the development of these themes in subsequent years and to identify patterns in the research aligned with the learning cycle components. Our approach is a neutral, automatic filtration and representation of available published literature based on specific criteria outlined in the next section. We use a representative sample of papers available through the OpenAlex API (Priem et al. [13]), which indexes 209 million works from various sources, including Crossref, PubMed, institutional repositories, and preprint repositories like arXiv, spanning 124,000 publishing venues. The results are conceptually organized and intended for both researchers and practitioners.

The rest of the paper is organized as follows: In the next section, we present an overview of the related works and position our paper in-light of the recent research. In the following section, we present our research article dataset and describe the filtering and preprocessing steps. We also briefly introduce the keyATM method, present the selected keywords, and discuss the resulting top words from the extracted topics. In the subsequent section, we conduct an in-depth analysis of the obtained topics and their interconnections. We establish the associations between the underlying learning cycle components and effective learning elements such as virtual and augmented reality, multimedia, gamification, and problem-based learning. Finally, we summarize our findings.

2. Related Work

Research on online learning has flourished in recent years, driven by the need to create adaptable and effective educational environments. A wide array of studies has focused on both theoretical frameworks and practical applications aimed at enhancing learning outcomes. Our work contributes to this growing body of research by focusing on the learning cycle framework and providing an automatic method for extracting and analyzing the trends and best practices from educational literature. To situate our work, we examine three relevant strands of research: learning cycles in education, topic modeling in educational research, and connections between innovative learning strategies and learning frameworks.

2.1. Learning Cycles in Education

The learning cycle, rooted in theories such as Kolb’s experiential learning model (Kolb [14]) and Bybee’s 5E instructional model (Bybee et al. [10]), has been an influential concept in education. As mentioned above, these cycles typically consist of stages like engagement, exploration, explanation, elaboration, evaluation. In the following, we will provide a more detailed overview of the learning cycle phases.

Engagement. Student engagement is defined in various ways. For instance, Hu et al. [15] described it as “the amount of effort dedicated to educational activities that bring out ideal performance”, while Lewis et al. [16] defined it as “the extent to which learners’ thoughts, feelings, and activities are actively involved in learning”. Wong and Liem [17] described learning engagement as “students’ psychological state of activity that enables them to feel activated, exert effort, and be absorbed during learning activities”. They distinguish between behavioral, cognitive, and emotional components of engagement based on established theories. According to the authors, learning engagement is a multilevel construct that varies depending on the learning context and time. In this sense, engagement is foundational to other stages, as it sustains the execution of learning activities. Lee et al. [18] identified several indicators of engagement: (1) behavioral—learning effort (e.g., self-completed units and time invested), participation in class activities (attendance and asking questions), and interaction (between learner and instructor); (2) cognitive—task solving (knowledge formation, and application, reflection on achievement), and learning management (self-direction, scheduling); and (3) emotional—learning satisfaction (interest, expectations, and enjoyment), sense of belonging (connection to the learning community), and learning passion (willingness to tackle challenges). Skilling et al. [19] connected engagement, motivation, and achievement, especially in secondary school mathematics, by measuring seven adaptive factors (self-efficacy, mastery orientation, valuing, persistence, planning, task management, and enjoyment) and five maladaptive factors (anxiety, failure avoidance, uncertain control, self-handicapping, and disengagement). Maroco et al. [20] developed a questionnaire to assess university student engagement, and Assunção et al. [21] confirmed that it produced reliable and valid data on academic engagement in higher education. Similarly, Tomás et al. [22] evaluated the reliability and validity of two additional self-assessment tools for quantifying student engagement.

Exploration. The exploration phase, as described by Bybee et al. [10], involves experimental activities, preliminary investigations, or using prior knowledge to generate new outcomes. Student activities may include making predictions, testing hypotheses, exploring alternatives, and generating new ideas. Depending on the subject, these activities can be implemented online through interactive applications, such as manipulatives (e.g., varying input parameters to produce different outcomes), simulations, and scenario-based explorations. These “hands-on” experiences have been shown to enhance learning success, as highlighted by Meylani et al. [23]. This phase is also closely related to experiential learning (see Morris [24]), where concrete experiences are central to the learning process.

Explanation. As outlined by Bybee et al. [10], this stage involves introducing new concepts or skills that serve as the theoretical foundation for solving the given problem or explaining the observed phenomena. In a fully online setting, where the instructor cannot easily adapt to individual learning preferences, these preferences should still be accounted for. Meylani et al. [23] reviewed studies showing that incorporating additional media formats can contribute to learning success. Learners should be provided with a variety of materials to understand new concepts, such as diagrams, videos, and text-based resources, as noted by Katsaris and Vidakis [25]. This ensures that diverse learning styles in e-learning are addressed.

Elaboration. According to Bybee et al. [10], elaboration deepens and broadens understanding, enhancing the application of new skills. In this phase, students should also have the opportunity to practice extensively and develop routine skills for solving problems related to newly acquired concepts. In asynchronous online learning, where direct instructor feedback is not available, self-testing with immediate feedback should be offered, as suggested by Meylani et al. [23]. To further engage students in practicing new concepts and gaining routine skills, gamified learning activities can be implemented. As noted by Demmese et al. [26], such gamified activities can motivate students to practice more by making the process enjoyable and encouraging them to solve additional exercises.

Evaluation. In this phase, ability assessment and the evaluation of student progress takes place. In online format, individual ability assessment is realized through quizzes with an end score. Meylani et al. [23] pointed out the importance of self-monitoring in an online framework. An overview of completed activities and attained test scores can be implemented in the form of a learning analytics dashboard, which have been shown to enable learners in making informed decisions (Susnjak et al. [27]).

Moreover, given that modern e-learning systems often incorporate adaptive or personalized features, we incorporate another component Evolution. This stage does not pertain to the learning cycle itself, but focuses on analyzing the usability of the learning system and suggesting improvements toward developing an adaptive, personalized e-learning system that accommodates individual learning preferences. The evolution section involves improving the e-learning environment based on direct feedback, questionnaires (such as the System Usability Scale, Lewis [28]), learning analytics, and incorporating personalized features. Meylani et al. [23] emphasized the importance of giving learners some control over the online resources available to them. Personalized recommendations, based on individual learning paths, can further enhance learning outcomes. As noted by Katsaris and Vidakis [25], personalized e-learning environments that adapt to individual learning preferences are on the rise and have the potential to optimally support individual learning processes. For example, Fatahi [29] proposed adapting e-learning environments according to learners’ emotions and personalities, while Khamparia and Pandey [30] reviewed methodologies for implementing e-learning adaptations based on automatic assessments of learning styles, highlighting their overall utility for improving learner success.

Numerous studies have highlighted the efficacy of these stages in promoting deep learning, engagement, and metacognitive development in various contexts, including inquiry-based and problem-based learning (Wilson et al. [31], Hmelo-Silver et al. [32]), and blended learning environments (Garrison and Kanuka [33]). Su et al. [34] used the 5E learning cycle to design an e-learning course and demonstrate its ability to enhance learning outcomes. While previous research has established the relevance of the learning cycle for guiding pedagogy, few studies have applied automated methods to examine its reflection in contemporary research literature, leaving a gap that our approach addresses.

2.2. Topic Modeling in Educational Research

Text mining and topic modeling techniques, such as Latent Dirichlet Allocation (LDA), have been widely used to extract thematic trends in educational research. For example, Costello et al. [35] applied LDA to explore the evolution of research topics in MOOC studies, while Pei et al. [36] used topic modeling to map key research areas in educational technology on multimodal learning analytics. Gurcan et al. [9] also applied LDA to identify emerging trends in e-learning. Topic modeling has proven effective in analyzing large corpora of research articles, helping to identify shifts in focus over time (Xiong et al. [37], Pei et al. [36], Mostafa [38]). Our work builds on these efforts by applying the keyword-based LDA of Eshima et al. [11], aligning extracted topics with the components of the learning cycle. This alignment provides a unique lens through which to analyze the thematic structure of recent online learning research, offering a new perspective that integrates educational theory with data-driven insights.

2.3. Connections Between Innovative Learning Strategies and Learning Frameworks

Several innovative learning strategies, such as virtual reality (VR), multimedia, gamification, and problem-based learning (PBL), have gained prominence in educational research and practice. These approaches have been found to enhance student engagement and outcomes by providing immersive, interactive, and learner-centered experiences (Dede [39], Hung et al. [40], Hamari et al. [41]). However, few studies have systematically analyzed how these elements relate to established theoretical learning frameworks like the learning cycle. For instance, Bellotti et al. [42] examined the role of gamification in the evaluation and exploration phases, while recent work by Yu and Xu [43] discussed the impact of VR on elaboration. Our study advances this line of research by systematically mapping these strategies onto all learning cycle components, thus offering a more holistic understanding of their integration into pedagogical design.

2.4. Contribution to the Literature

Our approach contributes to the literature in several ways. First, we offer a novel method for automatically extracting trends from relevant educational literature, applying LDA with carefully curated keywords to align topics with learning cycle components. Second, we provide insights into the temporal dynamics of these topics and shed light on how current research in online learning prioritizes different phases of the learning cycle. Finally, by mapping innovative learning strategies such as VR, multimedia, and gamification onto the learning cycle, we offer researchers and practitioners a new tool for designing and assessing learning activities in alignment with established educational theories.

This study is positioned at the intersection of automated text analysis, educational theory, and the study of online learning trends, thus filling an important gap in the current literature.

3. Data and Methods

In this section, we describe the search modalities used to retrieve the research articles considered for the topic analysis and provide information on filtering and preprocessing steps. Additionally, we briefly introduce the keyATM model, present the chosen keywords for the topics aligned with the learning cycle stages, and review the top words from the resulting topics.

3.1. Searching, Filtering, and Preprocessing Research Articles on Online Learning

Searching and filtering. We searched titles and abstracts of research articles published between 1 January 2019 and 31 March 2023, contained in the OpenAlex database (https://openalex.org/ accessed on 17 April 2023) using the keywords: “online learning”, “online teaching”, “e-learning”, “online course”, “online education”, “education technology”, “computer-assisted learning”, and “distance education”, with the R package openalexR (Massimo et al. [44]).

This search returned 106,032 articles (as of 17 April 2023). We then filtered the results to include only articles from publishers ranked in the top 20 on Google Scholar in education, educational technology, and educational psychology (https://scholar.google.com/citations?view_op=top_venues&hl=en&vq=eng_educationaltechnology (accessed on 17 April 2023)). This filtering process reduced the number of articles to 5474. Finally, we excluded articles with unavailable abstracts, resulting in a final dataset of 5089 articles for further analysis.

Figure 1 plots the number of articles in the selection by publication date (2023 is omitted; the dashed line marks the mean value), showing an upward trend in publications and a peak in 2021.

Preprocessing. To prepare the text corpus for topic analysis, we first combined the titles and abstracts of the selected articles. We then tokenized the content into individual words and pairs of consecutive words, applying standard preprocessing routines such as stemming and the removal of punctuation, numbers, symbols, and stopwords. Additionally, we removed terms that occurred in less than 0.1% or more than 50% of the articles to exclude terms that were either too rare or too common to be informative. After preprocessing, we obtained a corpus of 5089 documents with a vocabulary consisting of 15,022 features (words and word pairs).

This processed text corpus was then used to fit a keyATM model with six keyword-based topics, reflecting the engagement, exploration, explanation, elaboration, evaluation, and evolution components, as well as several keywordless topics that capture other thematic content from the articles.

3.2. Fitting the Keyword-Assisted Topic Model: keyATM

As mentioned in the previous section, our goal is to automatically extract topics structured around the learning cycle components. We assume that the titles and abstracts in our text corpus consistently reflect the topics covered in the main text of each article. Additionally, we assume that some of these topics align with the learning cycle components (keyword topics), while others may refer to content not directly related to the learning cycle (keywordless topics).

Topic Keywords. In our analysis, we align the keywords with the definitions and descriptions of the 5E stages, plus the evolution component discussed earlier, ensuring that the extracted topics reflect these learning cycle phases. The keywords, presented in

Table 1. Chosen keywords for the six keyword topics used to assist topic extraction.

Engagement	Exploration	Explanation	Elaboration	Evaluation	Evolution
engag	explor	explain	problem	assess	adapt
motiv	experi	instruct	practic	evalu	individu
emot	interact	knowledg	applic	test	improv
promot	activ	theori	solv	feedback	behavior

, were selected based on the definitions of the 5E stages from Eisenkraft [45], Duran and Duran [46], and Bybee et al. [10]. For the final component, evolution, we adopt descriptions of adaptive e-learning environments from Özcan Özyurt and Özyurt [47], El-Sabagh [48], and Kolekar et al. [49].

KeyATM model. To extract both types of topics—keyword topics aligned with the learning cycle and keywordless topics—we propose using the keyATM methodology of Eshima et al. [11]. This method is an LDA-based topic extraction algorithm that allows a user-specified focus in the form of topic keywords.

In keyATM, a corpus of D documents with vocabulary V is modeled to contain a total of K topics, of which $\tilde{K}$ are keyword-based and $K-\tilde{K}$ are keyword-free topics. Each keyword-based topic is associated with a set of $L_k$ keywords ${\mathcal{V}}_k=v_{k1},v_{k2},\dots ,v_{k{L}_k}$. In the topic model, each document d is a collection of $N_d$ words, represented by the set of words ${\mathcal{W}}_d=w_{d1},w_{d2},\dots w_{d{N}_d}$, and modeled in a Bayesian framework. Each $w_{di}$ is associated with a latent topic variable $z_{di}\in 1,2,\dots ,K$, fulfilling:

$$z_{di}\stackrel{indep.}{\sim }\mathrm{Categorical}\left({\theta }_d\right),$$

where ${\theta }_d$ is a K-dimensional vector of topic proportions for document d.

For a topic without keywords ($k\in \{\tilde{K}+1,\tilde{K}+2,\dots ,K\}$), the model assumes:

$$\left(w_{di}\right|z_{di}=k)\stackrel{indep.}{\sim }\mathrm{Categorical}\left({\varphi }_k\right),$$

where ${\varphi }_k$ is a K-dimensional vector of word frequencies for topic k. For keyword topics ($k\in \{1,2,\dots ,\tilde{K}\}$), a Bernoulli random variable $s_{di}$ with a success probability ${\pi }_k$ is introduced: $\left(s_{di}\right|z_{di}=k)\stackrel{indep.}{\sim }\mathrm{Bernoulli}\left({\pi }_k\right)$. If this variable equals to 0, then $\left(w_{di}\right|z_{di}=k)$ follows the same topic word distribution as for keyword-free topics. If this variable is equal to 1, then:

$$\left(w_{di}\right|z_{di}=k)\stackrel{indep.}{\sim }\mathrm{Categorical}\left({\tilde{\varphi }}_k\right),$$

where ${\tilde{\varphi }}_k$ is a V dimensional vector of word probabilities for the set of keywords of topic k, ${\mathcal{V}}_k$.

The prior distributions correspond to a Beta distribution for ${\pi }_k$ and Dirichlet distributions for ${\varphi }_k$ and ${\tilde{\varphi }}_k$ ($k=1,2,\dots ,K$). Furthermore, it is assumed that:

$${\theta }_d\stackrel{i.i.d.}{\sim }\mathrm{Dirchlet}\left(\alpha \right),$$

where $\alpha$ is a K-dimensional vector with components ${\alpha }_k$, following a Gamma distribution with distinct prior parameters for keyword-based and keyword-free topics.

Using the prior distributions, Eshima et al. [11] derived the conditional posterior distributions for $z_{di}=k$, $s_{di}$, and ${\alpha }_k$ and implemented a collapsed Gibbs sampling algorithm to sample from the posterior distributions. The estimation routine was provided in the R-package keyATM (see Eshima et al. [11]).

Choosing the number of topics. Using the keywords specified in Table 1, we fit the keyATM model to the data utilizing the keyATM package. However, we need to provide the total number of topics and the number of keyword-free topics as the input to the model estimation algorithm. The number of keyword topics is fixed at six, corresponding to the learning cycle. To choose the number of remaining keywordless topics, we applied a data-driven procedure based on the average topic coherence measure. This metric quantifies the quality of topics based on the co-occurrence of the top words within each topic (Thompson and Mimno [50], Selivanov et al. [51], Gurdiel et al. [52]) quantified the quality of topics based on co-occurrences of the top words constituting a topic. Specifically, we used the average mean–log-ratio topic coherence, defined for topic k with m top words $w_{k1},\dots ,w_{km}$ as:

$$co{h}_k=\sum _{i=1}^m\sum _{j<i}log{\displaystyle \frac{\#(w_{ki},w_{kj})}{\#\left(w_{ki}\right)}}+\epsilon ,$$

(1)

where $\#(·)$ counts the contexts containing the input (a word or a word pair), and $\epsilon$ is a smoothing parameter set to $1\times {10}^{-12}$. This metric quantifies how often the top m words in a topic k co-occur in the reference text corpus. It is justified by the observation that words with similar meanings tend to co-occur in the same contexts, with coherence positively associated with interpretability.

We alternated the total number of topics K in the range $[6,15]$, estimated the respective keyATM model, and calculated the average topic coherence for the keyword topics as defined in (1). Subsequently, we calculated the average coherence across the six keyword topics:

$$\overline{coh}={\displaystyle \frac{1}{k}}\sum _{k=1}^{\tilde{K}}co{h}_k.$$

(2)

The resulting average coherence for each K is shown in

Table 2. Average coherence for different total topic numbers, K, including the six prespecified keyword topics. The optimal topic number based on the average coherence criterion is $K=9$, implying that the number of keywordless topics should be set to 3.

K	$\overline{\mathit{coh}}$
6	−1.9798
7	−2.0108
8	−1.9869
9	−1.9657
10	−2.0116
11	−1.9689
12	−2.0301
13	−2.0082
14	−1.9821
15	−2.0677

. The optimal topic number maximizing the average coherence criterion in (2) is $K=9$, meaning that the optimal number of keyword-free topics should be set to three.

KeyATM model with the optimal number of topics. Based on the results of selecting the number of topics, we fit the keyATM model with the optimal number of topics, $K=9$ ($\tilde{K}=6$).

Table 3. Top 20 words for each keyword-based topic. Keywords matching the respective topic are marked with a checkmark. Keywords that appear in the top words of another topic are labeled with the corresponding keyword topic number.

1 Engagement	2 Exploration	3 Explanation	4 Elaboration	5 Evaluation	6 Evolution	7 Platform	8 Acceptance	9 Distance
self	interact [✓]	teacher	teach	assess [✓]	model	research	languag	covid
engag [✓]	cours	school	develop	group	cours	digit	mobil	pandem
motiv [✓]	social	mathemat	base	effect	system	higher	english	face
academ	design	instruct [✓]	design	test [✓]	mooc	review	accept	distanc
factor	activ [✓]	teach	practic [✓]	feedback [✓]	data	develop	intent	teach
effect	discuss	stem	process	evalu [✓]	learner	univers	model	univers
level	environ	servic	effect	perform	improv [✓]	articl	factor	institut
posit	experi [✓]	knowledg [✓]	applic [✓]	peer	base	analysi	perceiv	higher
signific	learner	develop	comput	read	open	paper	attitud	remot
relat	collabor	profession	method	result	qualiti	literatur	adopt	lectur
satisfact	particip	pre	skill	experiment	behavior [✓]	inform	influenc	challeng
relationship	support	children	problem [✓]	control	platform	context	toward	faculti
regul	blend	parent	paper	two	analysi	identifi	research	emerg
influenc	approach	ict	implement	base	individu [✓]	increas	foreign	experi [2]
efficaci	research	program	new	score	result	need	efl	time
perform	instructor	support	result	differ	propos	relat	comput	transit
result	engag [1]	integr	knowledg [3]	write	perform	futur	devic	survey
emot [✓]	class	classroom	integr	compar	user	profession	univers	chang
perceiv	explor [✓]	tpack	tool	intervent	manag	focus	usag	academ
differ	presenc	secondari	virtual	signific	adapt [✓]	includ	theori [3]	due

shows the top 20 words for each of the extracted topics, with the provided keywords marked by a checkmark. After reviewing the top words of the keywordless topics, we labelled them as “learning platforms”, “technical acceptance”, and “challenges in distance education” (hereafter referred to as “platform”, “acceptance”, and “distance”, respectively). As seen in Table 3, most of the provided keywords appeared among the top 20 words, confirming their effectiveness in assisting topic extraction. Keywords that also appeared in the top 20 words of a different topic were labeled with the number of the corresponding keyword topic. For example, “engag”, a keyword for topic 1, “engagement”, also appeared in the top 20 words for topic 2, “exploration”. This overlap was an exception, further suggesting that the keywords effectively supported the topic identification process.

Overall, the top words for the learning cycle topics, as revealed by the model, aligned well with the intended interpretation of the learning cycle components, indicating an acceptable model fit. In the next section, we provide an in-depth analysis of the extracted topics, focusing on their interrelations and associations with various effective learning elements.

4. Results and Discussion

In this section, we present and analyze the results of our automatic keyATM topic extraction and discuss their implications. We consider the overall topic proportions, the interrelations between the topics, and their connection to various effective learning elements.

4.1. Topic Proportions

In this part, we examined the topic proportions across the selected articles, as well as their evolution over time.

Figure 2 shows the expected topic proportions per paper. The largest proportions among the keyword-assisted topics were found in elaboration and exploration, followed by evolution and engagement. The explanation topic had the smallest expected proportion among the keyword-assisted topics. Among the keywordless topics, the first topic, referring to learning platforms, dominated. Overall, the topic proportions were balanced, indicating that most articles covered multiple topics with a high probability.

Figure 3 illustrates the dynamics of topic proportions over the publication years. Notably, we observed a slight increase in the literature focused on engagement in 2018 and 2022, as well as a significant rise in works addressing challenges in distance education during the pandemic restrictions of 2020–2021. Furthermore, in 2023, there was a notable increase in the literature devoted to the elaboration and evolution stages of the learning cycle, highlighting their growing importance in online learning applications.

In summary, our analysis of topic proportions reveals that elaboration and exploration are the most prominent topics, followed by evolution and engagement, with explanation having the smallest proportion. The temporal dynamics showed a recent rise in research on the engagement, elaboration, and evolution stages, alongside an increasing body of literature focused on the challenges of distance education.

4.2. Exploring the Interrelations Between Topics

In this section, we examine the interrelations between topics by investigating whether articles with a high proportion of one topic also tend to have a high proportion of another. This approach helps determine if there is a close relationship between topics. Additionally, we explore another method of analyzing these interrelations by examining citations among the papers. If papers with a high proportion of one topic frequently cite papers with a high proportion of another topic, this supports the existence of an interrelation.

To visualize these interrelations, we compute and plot a topic network with topics as vertices and two types of links as edges. The first type of link represents the expected topic proportion in documents with a high proportion of a given topic. The thickness of the link corresponds to this expected proportion: the thicker the line, the higher the proportion. For this network, we extracted the top 100 documents for each topic. The resulting network is shown in the left panel of Figure 4.

The second type of link is based on reciprocal citations among the top 100 documents for each topic. In this network, the thickness of the edges is adjusted according to citation intensity. The resulting network is displayed in the right panel of Figure 4.

In the left panel of Figure 4, we observe that articles with a high proportion of the engagement topic often also cover a significant proportion of exploration issues, and this topic is also associated with challenges in distance education. Similarly, articles with a high proportion of exploration tend to address themes related to learning platforms and technology acceptance issues. The explanation topic is linked to exploration and evolution, while articles focusing on evolution are heavily related to challenges in distance education. The evaluation topic exhibits strong interrelations, particularly with engagement, exploration, elaboration, the learning platforms topic, and challenges in distance education. Additionally, learning technology acceptance is closely tied to the evolution of learning systems and platform design.

In the citations network shown in the right panel of Figure 4, we observed that articles focused on exploration frequently cite works related to technology acceptance and other exploration-related papers. Documents with a predominant engagement focus tend to reference articles on evolution and other engagement-related works. Furthermore, articles discussing the adaptation of learning environments often cite works on distance education and research directions. Notably, articles on research directions are closely connected to evaluation and technology acceptance topics, with substantial references to works addressing challenges in distance education.

Additionally, we plot the network of the top 20 words for each topic based on the co-occurrences of the respective words in the documents. The resulting network is presented in Figure 5. The network graph shows that top words from the same topic tend to be interconnected but also relate to top words from other topics. Specifically, the top words of the evaluation topic (dark blue) are interconnected with those from the elaboration (light blue) and explanation (turquoise) topics. A noticeable interrelation is also seen between the top words of the engagement topic (red) and those of the elaboration stage (light green).

Overall, our examination of the interrelations between topics reveals that articles focused on engagement often overlap with exploration and address challenges in distance education, while evaluation shows strong ties with topics like engagement, elaboration, and learning platforms. The citation network revealed that articles on exploration frequently cite works on technology acceptance, while papers on engagement and evolution are interconnected through citations.

4.3. Exploring Topic Relations to Other Areas of Interest

Using the topics aligned with the learning cycle components and extracted by the keyATM model, researchers can explore other areas of interest by providing relevant keywords. These keywords can summarize the area of interest and allow for exploring the strength of its relation to the model’s topics. The strength of this relation could be proportional to the number of supplied keywords present in papers with a large proportion of each topic.

To illustrate this, we used “mathematics education” as an overarching area of interest, with the keyword “math” to explore its connection to the extracted topics. Figure 6 shows the results of this association analysis. In the left panel of Figure 6, a network graph illustrates how the keyword connects with the extracted topics. The analysis indicates a strong association with the engagement and explanation stages, while connections to the evaluation and evolution components are weaker. Additionally, the right panel of Figure 6 displays the titles of highly associated papers and their primary topics. This suggests that the primary challenges in math education are related to strategies for engagement and explanation.

Overall, the proposed model, which aligns topics with the components of the learning cycle, can be used to assess associations with other areas of interest based on keyword frequencies. This approach also helps identify research papers that may be relevant and valuable for the researcher.

4.4. Association Between Innovative Learning Elements and Topics

To provide concrete recommendations for developing successful online applications, we examine specific learning elements and their connections to the extracted topics. This approach enables instructors to select effective learning elements for each phase of the learning cycle. Our learning elements were aligned with the findings of Li et al. [53], which identified effective distance education elements not covered by the 5E learning principles. These elements include:

• VR: Virtual and Augmented reality (keywords: “realiti_augment”,“realiti_virtual”, “augment”, “realiti”)
• MM: Multimedia, including audio, video, animation content (keywords: “audio”,“video”, “anim”, “content”)
• GM: Gamification (keywords: “gamif”, “gamifi”, “game”, “play”, “game_studi”, “learn_gamif”, “gameplay”, “educ_game”, “game_engag”)
• PB: Practice-oriented, problem-based learning (keywords: “practic_learn”, “problem_base”, “problem_activ”, “problem_experi”, “practic_motiv”)

We analyzed the frequency of keywords associated with each learning element by reviewing the top 100 papers categorized according to the 5E principles and examining the top five papers from each of the 5E topics.

The first type of the considered effective learning elements—VR, or virtual and augmented reality—shows a strong association with the elaboration component (with a proportion of 0.77), as detailed in

Table 4. Topic distribution of effective learning elements.

	1 Engagement	2 Exploration	3 Explanation	4 Elaboration	5 Evaluation	6 Evolution	7 Platform	8 Acceptance	9 Distance
VR	0.00	0.00	0.05	0.77	0.03	0.01	0.10	0.03	0.00
MM	0.03	0.17	0.21	0.14	0.21	0.11	0.03	0.04	0.05
GM	0.32	0.06	0.10	0.20	0.19	0.03	0.02	0.06	0.02
PB	0.40	0.00	0.20	0.20	0.00	0.00	0.00	0.20	0.00

. Other stages of the learning cycle do not exhibit a significant connection with VR. In this regard, Nurbekova and Baigusheva [54] explored the effects of VR on students’ self perception, finding that VR made study materials more engaging and comprehensible, enhances the visibility of learning content, and provides better insights into theoretical concepts. Similarly, Zhao [55] reported that augmented reality enriches information delivery and improves student performance. Wolski and Jagodziński [56] assessed the impact of a virtual chemical laboratory and confirmed improvements in problem-solving. These findings were supported by meta-analyses conducted by Garzón and Acevedo [57] and Chang et al. [58].

The second group of effective learning elements—MM, or multimedia, audio, video, and animation—primarily influences the explanation and evaluation stages (with proportions of 0.21 and 0.21), but also affects exploration and elaboration. The analysis also highlights the importance of adapting multimedia technology to learners’ needs (with a weight of 0.11). For instance, Wang et al. [59] positively assessed the impact of multimedia-based educational enhancements on student abilities. Johnson and Cooke [60] evaluated different feedback modes related to the evaluation stage, while Van Laer and Elen [61] identified factors that enhance self-regulation in blended learning environments.

Gamification strategies are crucial for the engagement stage (with a proportion of 0.32), but they also enhanced the elaboration and evaluation stages (with proportions of 0.2 and 0.19). For example, Palaniappan and Noor [62] described a “gamification experience” involving competition, point earning, leaderboards, and badges, which significantly improved the learners’ performance by affecting motivation and perception. Kyewski and Krämer [63] and Albuquerque et al. [64] evaluated gamification elements such as badges and leaderboards, focusing on their impact on motivation and emotional factors. Meishar-Tal and Kesler [65] and Chen et al. [66] examined gamification for elaboration purposes, noting positive effects on motivation as well.

The final group of effective learning elements—PB, or problem-based methods—shows effects across engagement, explanation, and elaboration, with a primary focus on engagement (with proportions of 0.4, 0.2, and 0.2). For instance, Park and Yun [67] and Lazarides et al. [68] analyzed the relationship between motivational and cognitive factors, noting significant impacts. Rajabalee and Santally [69] studied the correlation between satisfaction and engagement in a project-based course. Yeoman and Wilson [70] emphasized the importance of situative design for learning outcomes in the explanation and exploration stages.

In summary, the effective learning elements aligned well with the learning cycle stages, as defined by the topic model. This alignment assists instructors in planning and designing innovative courses, integrating the learning cycle philosophy with these effective elements.

4.5. Conclusions

In this paper, we propose a novel method for analizing recent online learning literature by utilizing a keyword-based Latent Dirichlet Allocation, keyATM, to conduct a topic analysis for automatically identifying emerging trends through the lens of the learning cycle. By leveraging titles and abstracts from high-ranked educational journals, we assigned topic proportions to each article, where the topics corresponded to the stages of the learning cycle: engagement, exploration, explanation, elaboration, evaluation, and evolution. Our findings revealed temporal trends in the relative importance of each learning cycle component and the interconnections between the extracted topics and other areas of interest within current research.

Furthermore, we explored the relationship between these topics and key learning elements, such as virtual reality, multimedia, gamification, and problem-based learning. The analysis of effective learning elements highlights their alignment with specific stages of the learning cycle, offering valuable insights for course design and instruction. Virtual and augmented reality proved particularly effective in the elaboration phase, enhancing student engagement with theoretical concepts and improving material visibility. Multimedia tools extend their influence to both elaboration and evaluation stages, while also contributing to exploration. Gamification demonstrates broad utility across multiple stages, fostering motivation and active learning. Lastly, problem-based methods emphasize explanation and elaboration, underscoring the importance of situative design and cognitive engagement. These findings underscore the potential of integrating these elements into course planning to create innovative, learner-centered environments.

Overall, our approach allows innovative learning strategies and elements to be mapped within the framework of the learning cycle. The insights gained from this analysis can support researchers in designing targeted and effective learning activities that align with specific stages of the learning cycle.

The findings from this study offer significant practical value for educators and instructional designers by providing a structured approach to integrating emerging trends and effective learning elements within the learning cycle framework. By identifying the stages of engagement, exploration, explanation, elaboration, evaluation, and evolution in online learning literature, practitioners can better align their course design strategies with these key phases of the learning process. The insights into how tools like virtual reality, multimedia, gamification, and problem-based learning align with specific stages offer actionable guidance for enhancing instructional techniques. For instance, virtual reality can be leveraged to deepen conceptual understanding during the elaboration phase, while multimedia can play a critical role in both elaboration and evaluation. By using these findings, educators can create more dynamic and learner-centered environments, ensuring that innovative teaching methods are effectively incorporated into each stage of the learning cycle.

While our proposed method provides valuable insights into the alignment of online learning trends with the learning cycle framework, several limitations warrant consideration. First, the analysis relies on titles and abstracts, which may not fully capture the depth and nuance of the research articles, potentially overlooking important details. Expanding the analysis to include full texts could provide a more comprehensive understanding of topic proportions and trends. Additionally, the keyATM approach, while effective in aligning topics with learning cycle components, is sensitive to the selection of keywords, and further refinement of these keywords could enhance topic identification. Another limitation is the focus on high-ranked educational journals, which may exclude valuable contributions from other sources, such as conference proceedings or practitioner-oriented publications.

For future research, expanding the dataset to include a broader range of literature, refining the keywords and applying other topic modeling techniques could enhance the robustness of the findings. Additionally, studies exploring the evolving role of emerging technologies such as AI-driven personalized learning or adaptive learning platforms in the learning cycle would offer further insights. Finally, future work could investigate the practical application of these findings by analyzing the effectiveness of course designs that align with the identified trends and learning cycle components, providing empirical validation for the theoretical framework.