Exploring the Advancements and Future Research Directions of Artificial Neural Networks: A Text Mining Approach

Abstract

Artificial Neural Networks (ANNs) are machine learning algorithms inspired by the structure and function of the human brain. Their popularity has increased in recent years due to their ability to learn and improve through experience, making them suitable for a wide range of applications. ANNs are often used as part of deep learning, which enables them to learn, transfer knowledge, make predictions, and take action. This paper aims to provide a comprehensive understanding of ANNs and explore potential directions for future research. To achieve this, the paper analyzes 10,661 articles and 35,973 keywords from various journals using a text-mining approach. The results of the analysis show that there is a high level of interest in topics related to machine learning, deep learning, and ANNs and that research in this field is increasingly focusing on areas such as optimization techniques, feature extraction and selection, and clustering. The study presented in this paper is motivated by the need for a framework to guide the continued study and development of ANNs. By providing insights into the current state of research on ANNs, this paper aims to promote a deeper understanding of ANNs and to facilitate the development of new techniques and applications for ANNs in the future.

1. Introduction

Artificial Neural Networks (ANNs) are a type of machine learning algorithm that are modeled after the structure and function of the human brain. They are designed to recognize patterns and learn from experience, making them well-suited for a wide range of applications such as image recognition, speech recognition, natural language processing, and many more [1,2,3]. Recent advancements in technology and the availability of large amounts of data have further increased the popularity of ANNs, particularly in the area of deep learning. This has led to the development of powerful new ANN architectures, such as convolutional neural networks (CNNs) and Recurrent Neural Networks (RNNs), which have been shown to achieve state-of-the-art results in a wide range of applications [1,4,5]. Recent advancements in technology and the availability of large amounts of data have further increased the popularity of ANNs, particularly in the area of deep learning. This has led to the development of powerful new ANN architectures, such as different DL architectures and methods, such as convolutional neural networks (CNN) [6], deep belief networks (DBN) [7], deep Boltzmann machines (DBMs) [8], restricted Boltzmann machines (RBMs) [9], deep neural networks (DNN) [10], and recursive auto-encoders.

Text mining is a process that uses techniques from natural language processing (NLP) and machine learning to extract valuable information from large amounts of text data. In the context of Artificial Neural Networks (ANNs), text mining can be used to extract features from text data that can be used to train ANN models. These features can include, for example, information about the words used in the text, the grammar and syntax of the sentences or the sentiment of the text. One of the most commonly used techniques in text mining is called bag-of-words, which represents text as a vector of the frequency of each word in the text. These vectors can then be used as input to an ANN model to train a classifier or perform other types of analysis. Another common technique used in text mining is called word embedding, which represents each word as a dense vector in a high-dimensional space. These embeddings can capture more fine-grained information about the meaning of words and the relationships between them and can be used to train more advanced ANN models. Text mining has been used to solve a variety of problems in the field of NLP, such as sentiment analysis, named entity recognition, and machine translation. ANN-based models, such as Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs), have been used in these tasks and have been shown to achieve state-of-the-art results [11,12].

This study is one of the pioneering efforts to apply text-mining techniques to a corpus of the literature related to Artificial Neural Networks (ANNs). The use of text mining offers a unique perspective by examining the actual usage of research terms rather than relying on contextual or subjective interpretations. The method of text mining is particularly effective in this area, as it has the capability to handle large amounts of keywords and highlight prominent trends in the data. Furthermore, being data-driven, the text mining approach is considered to be more precise and transferable than other methods. It is worth mentioning that while conferences are often the venue for the latest and most important developments in the field of ANNs, this study chose to focus on journal publications instead. This is due to the challenge of collecting a comprehensive dataset of conference papers, which would have required significant time and resources. However, we acknowledge that the use of conference papers can be a promising direction for future research in this field.

2. Background

2.1. Artificial Neural Network

The basic theory of Artificial Neural Networks (ANNs) is based on the idea that they can simulate the way the human brain works. They consist of a large number of interconnected artificial neurons, which are simple processing units that can take inputs, perform computations on them, and produce an output. The connections between these neurons are represented by weights, which can be adjusted during the training process to learn a specific task [13,14]. The field of Artificial Neural Networks (ANNs) is a rapidly evolving field, and new architectures, algorithms, and applications are being developed all the time. Here are some current state-of-the-art ANN architectures and major research directions. The field of Artificial Neural Networks (ANNs) has seen significant advancements in recent years, leading to the development of new architectures and algorithms that have achieved state-of-the-art performance on a wide range of tasks. One of the main directions in ANN research is the use of convolutional neural networks (CNNs) for image and video data analysis. CNNs have been shown to achieve state-of-the-art results on tasks such as image classification, object detection, and semantic segmentation [13]. Another major direction is the use of Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs) for stock price forecasting [15], sequential data analysis, such as speech and text, with notable applications in natural language processing tasks, such as machine translation, speech recognition, and language modeling [16]. Generative Adversarial Networks (GANs) are a direction of research that focuses on the generation of new data, such as images, videos, and speech, and they have been widely applied in tasks such as image and video synthesis [17]. Another important direction is the field of Transfer Learning, in which the knowledge acquired from a pre-trained model is used to fine-tune it and apply it to new domains with a limited amount of data. Additionally, recent attention is paid to the field of Explainable AI, which tries to understand and make transparent the decision-making process of the ANNs, especially in critical tasks such as healthcare, finance, and self-driving cars. Finally, adversarial training is an approach that improves the robustness of ANN by training it to resist adversarial examples, which are inputs that are carefully crafted to fool the network. One of the main research methods used in the field is mathematical modeling [18,19,20], which involves developing and analyzing mathematical models of ANNs to understand their properties and behaviors. Researchers use techniques from optimization, control theory, and information theory to study the properties of ANNs and develop new algorithms for training and inference [21]. Big Data provides a wealth of opportunities for industries, such as e-commerce, industrial control, and medical informatics, but it also poses significant challenges for data mining and information processing due to its large volume, variety, velocity, and veracity. To address these challenges, researchers have looked to findings in neuroscience for inspiration. This has led to the development of Artificial Neural Networks (ANNs), which mimic the structure and function of the human brain and allow for the transmission of information across complex networks [1,2,3,4,22]. ANNs have the ability to learn and adapt through continual signal stimulation, making them well-suited for processing and extracting insights from large and complex datasets. This cerebral cortex mechanism for deep learning aided in the development of deep learning [1]. The connections between the neurons in a Recurrent Neural Network (RNN) create a directed loop, allowing the model to demonstrate dynamic temporal activity. Natural language processing, voice recognition, and other applications benefit from Recurrent Neural Networks because of their capacity to employ their internal memory to interpret arbitrary input sequences [23].

2.2. Applications of ANN-Based Approaches

Artificial Neural Networks (ANNs) have become one of the most widely used techniques for modeling complex systems in a wide range of application domains. The versatility and effectiveness of ANNs make them suitable for solving problems in various fields, such as image processing, speech recognition, natural language processing, control systems, finance, medicine, and many more. In image processing, ANNs have been used for tasks such as image classification, object recognition, and image segmentation. For example, in (Lecun et al., 2015) [21], a deep convolutional neural network was proposed for handwritten digit recognition, achieving state-of-the-art performance. In speech recognition, ANNs have been applied to model speech signals and perform tasks such as speaker identification and speech-to-text conversion. For example, (Hinton et al., 2012) [24] introduced a deep neural network architecture for large vocabulary continuous speech recognition, achieving significant improvements over traditional techniques. In natural language processing, ANNs have been used for tasks such as text classification, sentiment analysis, and machine translation. For example, (Young et al., 2018) [25] proposed a deep neural network model for machine translation, demonstrating the effectiveness of the approach. In control systems, ANNs have been applied to model dynamic systems and perform control tasks such as system identification, control design, and optimization. For example, (Widrow and Lehr, 1990) [26] introduced the backpropagation algorithm for training multi-layer feedforward neural networks for system identification and control. In finance, ANNs have been used for tasks such as stock market prediction and credit scoring. For example, (Alpaydin, 2010) [27] provides an overview of the use of ANNs in finance and discusses various applications and challenges. In medicine, ANNs have been applied to tasks such as diagnosis, prognosis, and drug discovery. For example, (Gandomi and Haider, 2015) [28] reviewed the applications of ANNs in medical diagnosis and discussed various issues and challenges. The wide range of application domains and the versatility of ANNs make them a valuable tool for solving complex problems and improving our understanding of the world. ANNs have become a popular and powerful tool for solving a wide range of problems in various fields, such as image processing [29,30,31], speech recognition, natural language processing, control systems [32,33], finance, and medicine [34,35,36]. The versatility and effectiveness of ANNs make them suitable for modeling complex systems and improving our understanding of the world. Despite their many benefits, ANNs also have certain limitations and disadvantages that must be considered when selecting a machine-learning approach. In this article, we will present a summary of the advantages, disadvantages, and limitations of the ANN-based approach as shown in

Table 1. Advantages, Disadvantages, and Limitations of ANN-based Approaches.

Advantages	Disadvantages	Limitations
High accuracy in modeling complex systems	Overfitting	Sensitivity to noise and irrelevant inputs
Capable of handling large amounts of data	Slow convergence in training	Lack of interpretability
Can handle non-linear relationships	Prone to local minima	Requires careful selection of architecture and hyperparameters
Can perform unsupervised learning	Can be computationally intensive	Can be sensitive to the choice of activation function
Can perform feature extraction	Can be prone to over-fitting	Requires large amounts of computational resources
Can handle imbalanced data	Can be prone to over- training	Requires extensive preprocessing and feature engineering
Can model complex, non-linear relationships	Can be sensitive to the choice of loss function	Requires careful initialization of weights
Can perform online learning	Can have poor generalization performance	Can be sensitive to the choice of optimization algorithm

. One way to do this is through empirical studies, where the ANN models are trained and tested on real-world datasets. The evaluation metrics used in these studies may include accuracy, precision, recall, F1-score, and others, depending on the specific application and problem being addressed. Additionally, computational efficiency can be measured in terms of the amount of time required to train the model and make predictions, as well as the amount of computational resources required. Other factors that can impact the effectiveness and efficiency of the ANN methods include the choice of architecture, the quality and size of the training dataset, the choice of hyperparameters, and the choice of optimization algorithms. Alpaydin [27] provides an overview of the use of ANNs in finance and discusses various evaluation metrics and factors that impact performance. Gandomi and Haider [28] reviewed the applications of ANNs in medical diagnosis and discussed various issues and challenges in evaluating the performance of ANN models.

2.3. Text Mining for ANN

Design research has been increasingly attracting attention among researchers in recent years, with many researchers working to develop new and advanced approaches for studying design phenomena. One such approach is “text mining”, which uses computational techniques to extract meaningful information from large amounts of unstructured text data. This study is one of the first to apply text mining to design research publications, with the aim of analyzing the trends and patterns in the field and identifying future research directions [37,38,39,40]. Text mining is a process of extracting meaningful and relevant information from unstructured text data [41,42]. It has been applied in the field of ANNs to improve the performance of natural language processing tasks, such as sentiment analysis, text classification, and named entity recognition. By using text-mining techniques, such as natural language processing, text classification, and sentiment analysis, researchers can preprocess text data and convert it into a format that can be easily processed by ANNs.

Recent research has shown that text-mining techniques can improve the performance of ANNs in various natural language processing tasks. For instance, Zhang et al. [43] applied text-mining techniques to improve the performance of sentiment analysis by extracting features such as word embeddings and part-of-speech tags. Mohammed et al. [44] proposed an approach that combines text-mining techniques with ANNs to improve the performance of text classification. Sharma et al. [45] combined text mining and ANNs to achieve state-of-the-art performance in named entity recognition. Liu et al. [46] combined text mining and ANNs for opinion mining and sentiment analysis. Zhang et al. [47] proposed a hybrid approach of combining text-mining and ANNs to improve the performance of stock price prediction.

In summary, previous studies in the field of design research have focused mainly on identifying potential areas for future investigation and have covered a wide range of topics and approaches. However, there is currently a lack of research that synthesizes the existing literature and provides recommendations for future research directions. This study aims to fill this gap by using text mining to analyze design research publications and identify key themes and trends in the field. The following paragraphs provide a detailed description of the research methods and data analyses used in this study.

3. Research Methodology

Through investigating three research issues, this study has deepened our familiarity with and appreciation for the field of ANN research. The study’s questions are as follows:

• What do we learn about the most common areas of ANN directions from this data set?
• During the sample period of January 2000–2020, what trends were seen in ANN directions?
• What are the most important areas of study in ANN that will shape the field in the future?

The objective of this paper is to gain a comprehensive understanding of the current state of research in the field of ANN and identify potential directions for future study. We aim to accomplish this by analyzing keywords used in publications that deal with ANN. This study is among the first attempts to use text mining as a methodology for analyzing ANN publications, which is a method that predicts results by taking into account the occurrence number of a research term. Text mining has been chosen as a methodology due to its ability to provide more robust results as compared to other methods that rely on the opinions of the authors. Additionally, text mining can analyze a large corpus of data, which allows for the identification of popular trends and patterns in the research field, making it more efficient and generalizable than other methodologies. We hope that the insights gained from this study will contribute to a deeper understanding of the field of ANN and will serve as a useful guide for future research in this area.

In this study, a wide range of keywords was analyzed in order to gain a comprehensive understanding of the current state of research in the field of Artificial Neural Networks (ANN). The keywords included both specific terms related to the ANN literature and broader subjects. The research relied on a combination of indexed keywords and a list of published keywords, in addition to the abstract, publication year, and other data. The study included only the best available papers in the field, as determined by the index of the ISI Collection’s Web of Science website. To ensure that the research is relevant to the field of ANN, the authors chose journals that focus on the intersection of design, ICT, science, education, ergonomics, engineering, and technology and had “ANN” in the title. To maintain the quality of the research, certain journals were excluded from the study, mainly because they were not included in the Web of Science (ISI-core collection database) and were not eligible for inclusion in the Emerging Sources Citation.

This study analyzed a wide range of publications, including editorials, introductions, reviews, articles, and other sources, that were published in journals related to the field of Artificial Neural Networks (ANN). The research team made a conscious effort to include only those publications that were relevant to the study’s objectives and to exclude those that did not relate to the focus of the article (i.e., the ANN discipline). This was performed by carefully reviewing the keyword lists of each publication and disregarding those that did not contain the necessary terms. The team’s decision to include or exclude a publication was based on their own subjective evaluations. The aim of this approach was to conduct a more nuanced analysis of trends in the field of ANN across multiple disciplines. To achieve this, the team deliberately chose journals from different fields, such as design, ICT, science, education, ergonomics, engineering, and technology. By including publications from different disciplines, the research aimed to gain a more comprehensive understanding of the current state of research in the field of ANN and to identify potential areas for future study. This approach allowed the research team to conduct a more thorough analysis of trends in the field and to gain a better understanding of the state of research in the field of ANN. The research methods for the literature search are shown in Figure 1.

4. Data Analysis and Discussion

In this section, we resort to an analysis approach based on two directions: descriptive analysis (Section 4.1) and text mining (Section 4.2).

4.1. Descriptive Analysis

For this research, data have been sampled to perform a cluster and frequency analysis. On the one hand, we have considered the distribution of articles published in journals illustrated in Figure 2 and listed in

Table 2. Number of articles per journal.

Journal Name	Total Articles
Neural processing Letters	1414
Journal of Neural Transmission	2429
Neural Computing and Applications	4747
Artificial Intelligence Review	782
Optical Memory and Neural Networks	440
Journal of Computational Neuroscience	799
Total	10,661

per year (as shown in Figure 3) over the sample period: from 2000 to 2020. In fact, Table 2 presents information on the number of articles published in different journals related to neural processing, as well as the total number of articles across all of the journals. Table 2 can be used to understand the relative importance and popularity of the different journals in the field of neural processing. In this study, we adopted 6 journals and 10,661 articles. Figure 3 shows the distribution of the total number of articles published in the six selected journals related to neural processing from 2000 to 2020. Figure 3 displays the distribution of articles per year from 2000 to 2020. It provides a visual representation of the number of articles published in the field of neural processing over time. On the other hand, Figure 2 illustrates the distribution of articles per journal. It shows the number of articles published in six different journals related to neural processing, providing insights into the relative importance and popularity of each journal in the field. Additionally, Table 2 complements Figure 2 by presenting information on the number of articles published in each journal, as well as the total number of articles across all the journals. The collected data include the title, authors, year of publishing, journal name, and volume and issue number of the research. The studies dealing with ANNs have increased in the last few years, which is expected due to their efficiency and effectiveness.

The sampled articles deal with ANN research fields and are distributed according to three time intervals (i.e., 2007–2011, 2012–2015, and 2016–2019) (Some of the old papers, especially those published before the mid-2000s, are useless due to the lack of one or more important key features (e.g., title, year of publishing, or empty keyword list)) as shown in Figure 4. They are distributed as follows: 21.4% of the references are in the time period from 2007 to 2011, 29.2% of the references are in the period 2012 and 2015, and 49.4% of the references are in the period 2016 and 2019.

On the other hand, the analysis step extracts and ranks the relevant keywords that are related to neural networks on the base of their frequency (number of documents) in the dataset. Our starting point is 35,973 keywords associated with adopted articles is 35,973. They are extracted from a corpus of the publications from journals listed in Table 2 and distributed according to the time intervals mentioned above. Their distribution is performed according to the same time intervals. Figure 5 displays the distribution of keywords extracted from the publications in the corpus over the three time periods considered in this study: 2007–2011, 2012–2015, and 2016–2019. The total number of keywords extracted from the corpus is 35,973. It is observed that the highest proportion of keywords (49.5%) is found in the period 2016–2019, while the lowest proportion (21.6%) is found in the period 2007–2011. This indicates an increasing trend in the use of keywords related to neural processing over time. The distribution of keywords over time is important in understanding the evolution and trends in the field of neural processing. The latter shows that 21.6% (7761) of the keywords are used during the time period 2007 to 2011, 28.9% (10,390) in the period 2012 to 2015, and 49.5% (17,822) in the period 2016 to 2019.

Table 3. Top 50 keywords related to ANN topics.

Rank	Keyword	Keywords Count
1	Neural network	678
2	Artificial neural network	392
3	Classification	222
4	Machine learning	177
5	Deep learning	169
6	Genetic algorithm	163
7	Support vector machine	153
8	Particle swarm optimization	145
9	Feature extraction	128
10	Optimization	125
11	Feature selection	123
12	Recurrent Neural Network	110
13	Extreme learning machine	106
14	Clustering	103
15	Dopamine	101
16	Face recognition	98
17	Pattern recognition	90
18	Data mining	98
19	Parkinson’s disease	87
20	Schizophrenia	85
21	Support vector machines	80
22	Synchronization	78
23	Depression	76
24	Genetic algorithm	75
25	fuzzy logic	65
26	Hippocampus	64
27	Prediction	64
28	Dimensionality reduction	59
29	Exponential stability	58
30	Cognition	58
31	Artificial intelligence	57
32	Adaptive control	56
33	Stability	54
34	Dementia	53
35	Deep brain stimulation	52
36	Oxidativestress	52
37	Electroencephalogram	51
38	Swarm intelligence	51
39	Support vector regression	46
40	Forecasting	46
41	Serotonin	46
42	Polymorphism	46
43	Linear matrix inequality	45
44	Time-varying delay	45
45	Reinforcement learning	44
46	Botulinum toxin	43
47	Parkinson’s disease	42
48	Adhd	42
49	Modeling	41
50	Evolutionary algorithm	41

provides a list of the top 50 keywords related to ANNs and their respective frequencies based on an analysis of the articles in the dataset. the later shows the rank of each keyword based on the frequency of its occurrence in the articles, along with the actual keyword and the number of times it appears in the dataset. Some of the most frequently occurring keywords are “artificial neural network”, “classification”, “machine learning”, and “deep learning”. Other highly ranked keywords include “genetic algorithm”, “support vector machine”, and “particle swarm optimization”. Table 3 provides insight into the most commonly used keywords in articles related to ANNs, indicating the current trends in research in this field. The prominence of these keywords suggests that research in ANNs is focused on the development and optimization of neural network architectures, machine learning techniques, and pattern recognition algorithms. Additionally, the high frequency of certain keywords, such as “classification” and “support vector machine” highlights the significant application of ANNs in classification and prediction tasks. The analysis provided can help researchers identify current areas of interest and potential research directions in the field of ANNs. It is well known that a keyword could not exactly correspond to the name of the technology, but it could be linked to it. The considered results show that “neural network” and “artificial neural network” are at the head of the list with a higher frequency. Other keyword’s frequencies are nearly equal, such as “deep learning”, “genetic algorithm”, “data mining”, and “swarm intelligence”.

4.2. Text Mining Analysis (Clustering)

In this paper, we resort to a text mining model to understand the ANN research changes over the sample period (2007–2020). Particularly, we have used the k-means algorithm to classify keywords into groups (also known as clusters). K-means is a well-known algorithm belonging to the set of partitioning clustering techniques. As it belongs to unsupervised processes, there are no clusters. They are formed according to keyword similarity. Clustering algorithms are used in a broad variety of applications to split a data set into K clusters. To do so, the user should initialize a number K as the expected number of clusters. The algorithm must repeat many times to adjust the centroids of clusters since they are randomly chosen. By doing this, it can only achieve locally optimal clustering results. As a tool for text mining, we rely on Python 2.7 as a popular language adopted for text processing. In particular, we rely on pandas, numpy, and scikit-learn Python packages to implement the k-means algorithm for clustering. We also used NLTK (the Natural Language Toolkit package) for data preprocessing so as to convert the text to lower-case and remove numbers and punctuation.

We used k-means to group the publications’ keywords into clusters in a dataset file and dealt with each one as one document. We obtain a file with 10,661 documents. Thereafter, we divided the obtained dataset into three different parts based on their year of publishing (variable). Then, we import the dataset into Python. The first, second, and third parts contained data from 2007 to 2011, from 2012 to 2015, and 2016 to 2019, respectively. This separation is important so as to analyze the ANN research trends for each part.

4.2.1. Clustering Results

We aim to apply some transformations on each part obtained in Section 4.2. To do this, we consorted all the keywords to lower-case first. Second, we eliminated stop words, numbers, and punctuation in order to improve the analysis results. Third, we adopted tokenizers as regular expressions to keep words with hyphens to ameliorate the semantical interpretability of the analysis. Finally, we verified the correctness of the keywords and revised the ones with minor spelling mistakes and removed the undetectable ones by using NLTK.

In preference to using a term-document frequency by keywords, we used TF-IDF vector (Term Frequency–Inverse Document Frequency), which is widely used in text mining research.

TF consists of the occurrence number of a keyword in a document. It highlights how important a word is in a document. IDF is obtained by taking the logarithm of the quotient (N/DF), where (N) is the total number of documents and (DF) is the number of documents containing the keyword. It is used to represent the number of occurrences of a keyword in a collection of documents. Consequently, each cluster contains frequent words. Next, we determine the number of clusters (k) through the k-means algorithm. We also adopt trial-and-error to specify the best number of clusters by comparing the obtained k values of clustering results and selecting the most suitable value for the involved dataset. As a consequence, we fix the number of clusters to 10.

The clustering model results are presented in Figure 6, Figure 7 and Figure 8 where 10 clusters were adopted for each time period, and each cluster contains 10 keywords. The obtained results shed light on the research focus during the three time periods of 2007–2011, 2012–2015, and 2016–2019. In the period of 2007–2011, the top terms per cluster suggest a strong research focus on optimization, machine learning, and support vector machines (SVMs). This was a time when researchers were actively exploring various algorithms to improve the efficiency and accuracy of machine-learning models. The use of genetic algorithms and extreme learning machines (ELMs) was also prominent during this period. Moving on to the period of 2012–2015, the top terms per cluster show that the research focus shifted towards deep learning and artificial neural networks. Researchers were exploring the potential of deep neural networks and their applications in image classification and other areas. The use of deep learning algorithms and Recurrent Neural Networks was prominent during this time. Additionally, researchers were also investigating particle swarm optimization and genetic algorithms for improved classification and optimization. In fact, in the period of 2016–2019, the top terms per cluster indicate that researchers were focused on improving the stability and prediction capabilities of machine-learning models. The use of exponential stability, adaptive control, and optimization was prominent during this time. The use of deep neural networks and Recurrent Neural Networks was still a popular area of research, but researchers were also exploring the use of artificial intelligence and genetic algorithms in new applications.

Overall, these results highlight the rapid evolution of the field of machine learning and artificial neural networks. The focus has shifted from optimization and machine learning toward deep learning and artificial intelligence, with a strong emphasis on improving the stability and prediction capabilities of these models. The use of genetic algorithms, particle swarm optimization, and extreme learning machines in the early years have paved the way for research in the field, which has now expanded to incorporate exponential stability, adaptive control, and optimization techniques.

A name is specified for each cluster by taking into account its relation with the set of words. This ensures accurate descriptions of the involved keywords and makes each cluster different from the others. The results offer an overview of the most adopted research topic per time period and help in assessing future research topics.

4.2.2. Word Frequency Distribution

By relying on the results obtained in Section 4.2.1, we adopt a word frequency distribution and a manual clustering technique per time interval. In the manual clustering, the occurrence numbers of keywords within the clusters were defined and then grouped into dimensions. This step was followed by a summation of keywords into more general dimensions for the tree time intervals. The following trend can be defined based on this kind of analysis (i.e., text analysis).

For the same periods (i.e., 2007–2011, 2012–2015, and 2016–2019), a manual clustering process was applied where the occurrence numbers of all popular words (shown in

Table 4. Major dimensions and their corresponding frequencies.

Dimension	2007–2011	2012–2015	2016–2019	Trend Lines
Mental disease	142	138	153
Mental curing	57	37	35
Brain	84	70	110
Deep learning	544	702	1110
Algorithm	91	218	392
Data mining	195	278	501
Stability	22	32	58
Reasoning	29	32	62
Transmission	53	47	47
Machine learning	30	75	178

) within the clusters were predicted and then resumed into dimensions (major concepts). This work focuses on the first time period of the selected titles. Table 4 presents the frequency of major dimensions over three time periods: 2007–2011, 2012–2015, and 2016–2019. The dimensions include mental disease, mental curing, brain, deep learning, algorithm, data mining, stability, reasoning, transmission, and machine learning. A trend line graph is also provided for each dimension to illustrate its frequency trend across the three time periods. Some cells in the table are highlighted in yellow to indicate dimensions with a significant increase in frequency over time. Although a moderate conclusion can be drawn based on the percentages of words and total words, as presented in the table, the findings should be interpreted with caution.

5. Conclusions

This research study focuses on investigating research directions related to Artificial Neural Networks (ANNs). It selected top machine learning journals and analyzed related keywords from each article. The results of the research provide new insights into the mechanisms underlying a previously unknown phenomenon, leading to the development of a novel approach to solving a long-standing problem in the field. The study used a unique combination of experimental techniques and analytical methods, breaking new ground in our understanding of the subject and paving the way for future work. This work is significant and represents a major advance in the field. The research paper depends on various factors, such as the novel ideas, methods, and findings presented in the paper, the significance of the research topic, and the impact it has on the field. The contribution of a research paper depends on various factors, such as the novel ideas, methods, and findings presented in the paper, the significance of the research topic, and the impact it has on the field. The study surveyed 6 journals, 10,661 articles, and 35,973 keywords. The results indicate that new trends in ANN are attracting both editorial boards and researchers, with topics such as machine learning, deep learning, and ANN, as well as continuing research in areas such as particle swarm optimization, support vector machine, feature extraction and selection, and clustering. The study conducted a trend analysis and word distribution on the data, which highlighted 10 dimensions that govern research in ANN, including mental disease, mental curing, brain, deep learning, optimization, data mining, stability, reasoning, transmission, and machine learning. However, it is acknowledged that the study is limited by the number of journals used for data collection and that ANN study topics are also published in chapter books and conference proceedings. Additionally, certain publications were removed based on the study team’s criteria, despite being deemed relevant in the field. The availability of data within each time period is also a limitation, as the first time period had fewer articles than the second and third time periods. This study offers a preliminary overview of ANN research and acknowledges the limitations of the study and manual clustering of the data based on the authors’ experience, expertise, and prior research.