Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
3.7
2.3
Journals
Algorithms
Special Issues
Natural Language Processing with Tsetlin Machines and Deep Neural Networks
share announcement

Natural Language Processing with Tsetlin Machines and Deep Neural Networks

A special issue of Algorithms (ISSN 1999-4893). This special issue belongs to the section "Evolutionary Algorithms and Machine Learning".

Deadline for manuscript submissions: closed (30 November 2022) | Viewed by 7776

Special Issue Editors

Prof. Dr. Ole-Christoffer Granmo

E-Mail Website
Guest Editor
Department of Information and Communication Technology, University of Agder, 4879 Grimstad, Norway
Interests: learning automata; bandit algorithms; Tsetlin machines; Bayesian reasoning; reinforcement learning; computational linguistics
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Lei Jiao

E-Mail Website
Guest Editor
Department of Information and Communication Technology, University of Agder, 4879 Grimstad, Norway
Interests: Tsetlin machines; learning automata; reinforcement learning; stochastic processes
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Vladimir Zadorozhny

E-Mail Website
Guest Editor
School of Computing and Information, University of Pittsburgh, Pittsburgh, PA 15260, USA
Interests: information integration; data fusion and sense-making; complex adaptive systems; scalable data streams and Tsetlin machines
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Morten Goodwin

E-Mail Website
Guest Editor
Centre for Artificial Intelligence Research (CAIR), University of Agder, Jon Lilletuns Vei 9, N-4879 Grimstad, Norway
Interests: computational linguistics; deep learning; Tsetlin machines; reinforcement learning
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

The emerging paradigm of Tsetlin machines is allowing a fundamental shift from arithmetic-based to logic-based machine learning. At the core, finite-state machines learn patterns using logical clauses, and these constitute a global description of the task learnt. The paradigm has enabled competitive accuracy, scalability, memory footprint, inference speed, and energy consumption across diverse tasks, including classification, convolution, regression, natural language processing (NLP), and speech understanding.  

Tsetlin machines have been particularly successful in NLP. The pioneering NLP approaches use Boolean bag-of-words to represent natural language and logical clauses to capture textual patterns. Recent work addresses text classification, word-sense disambiguation, semantic relation analysis, novelty detection, and aspect-based sentiment analysis.  

Deep neural networks have been widely recognized and employed for various NLP applications. A combination of Tsetlin machine and deep neural networks may solve NLP tasks in a more efficient and transparent way. 

In this Special Issue, we invite papers that advance the state-of-the-art in NLP with Tsetlin machines and deep neural networks. The issue will cover both methodology advances and novel applications.

Prof. Dr. Ole-Christoffer Granmo
Dr. Lei Jiao
Prof. Dr. Vladimir Zadorozhny
Prof. Dr. Morten Goodwin
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Algorithms is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Tsetlin machine NLP methodology:
    • first-order logics
    • embedding
    • boolean representations
    • clustering
    • interpretation
    • convolution
    • novelty detection
    • knowledge representation
    • Hawkes processes
    • attention
  • Tsetlin machine NLP applications:
    • sentiment analysis
    • question-answering
    • word-sense disambiguation
    • speech understanding
    • chatbots
    • explainable machine learning
    • continuous interpretation of document streams

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

18 pages, 958 KiB  
Article
Tsetlin Machine for Sentiment Analysis and Spam Review Detection in Chinese
by Xuanyu Zhang, Hao Zhou, Ke Yu, Xiaofei Wu and Anis Yazidi
Algorithms 2023, 16(2), 93; https://doi.org/10.3390/a16020093 - 8 Feb 2023
Cited by 1 | Viewed by 2009
Abstract
In Natural Language Processing (NLP), deep-learning neural networks have superior performance but pose transparency and explainability barriers, due to their black box nature, and, thus, there is lack of trustworthiness. On the other hand, classical machine learning techniques are intuitive and easy to [...] Read more.
In Natural Language Processing (NLP), deep-learning neural networks have superior performance but pose transparency and explainability barriers, due to their black box nature, and, thus, there is lack of trustworthiness. On the other hand, classical machine learning techniques are intuitive and easy to understand but often cannot perform satisfactorily. Fortunately, many research studies have recently indicated that the newly introduced model, Tsetlin Machine (TM), has reliable performance and, at the same time, enjoys human-level interpretability by nature, which is a promising approach to trade off effectiveness and interpretability. However, nearly all of the related works so far have concentrated on the English language, while research on other languages is relatively scarce. So, we propose a novel method, based on the TM model, in which the learning process is transparent and easily-understandable for Chinese NLP tasks. Our model can learn semantic information in the Chinese language by clauses. For evaluation, we conducted experiments in two domains, namely sentiment analysis and spam review detection. The experimental results showed thatm for both domains, our method could provide higher accuracy and a higher F1 score than complex, but non-transparent, deep-learning models, such as BERT and ERINE. Full article
(This article belongs to the Special Issue Natural Language Processing with Tsetlin Machines and Deep Neural Networks)
►▼ Show Figures

Figure 1

21 pages, 7405 KiB  
Article
Data Augmentation Methods for Enhancing Robustness in Text Classification Tasks
by Huidong Tang, Sayaka Kamei and Yasuhiko Morimoto
Algorithms 2023, 16(1), 59; https://doi.org/10.3390/a16010059 - 16 Jan 2023
Cited by 4 | Viewed by 2687
Abstract
Text classification is widely studied in natural language processing (NLP). Deep learning models, including large pre-trained models like BERT and DistilBERT, have achieved impressive results in text classification tasks. However, these models’ robustness against adversarial attacks remains an area of concern. To address [...] Read more.
Text classification is widely studied in natural language processing (NLP). Deep learning models, including large pre-trained models like BERT and DistilBERT, have achieved impressive results in text classification tasks. However, these models’ robustness against adversarial attacks remains an area of concern. To address this concern, we propose three data augmentation methods to improve the robustness of such pre-trained models. We evaluated our methods on four text classification datasets by fine-tuning DistilBERT on the augmented datasets and exposing the resulting models to adversarial attacks to evaluate their robustness. In addition to enhancing the robustness, our proposed methods can improve the accuracy and F1-score on three datasets. We also conducted comparison experiments with two existing data augmentation methods. We found that one of our proposed methods demonstrates a similar improvement in terms of performance, but all demonstrate a superior robustness improvement. Full article
(This article belongs to the Special Issue Natural Language Processing with Tsetlin Machines and Deep Neural Networks)
►▼ Show Figures

Figure 1

13 pages, 808 KiB  
Article
Enhancing Attention’s Explanation Using Interpretable Tsetlin Machine
by Rohan Kumar Yadav and Dragoş Constantin Nicolae
Algorithms 2022, 15(5), 143; https://doi.org/10.3390/a15050143 - 22 Apr 2022
Viewed by 2250
Abstract
Explainability is one of the key factors in Natural Language Processing (NLP) specially for legal documents, medical diagnosis, and clinical text. Attention mechanism has been a popular choice for such explainability recently by estimating the relative importance of input units. Recent research has [...] Read more.
Explainability is one of the key factors in Natural Language Processing (NLP) specially for legal documents, medical diagnosis, and clinical text. Attention mechanism has been a popular choice for such explainability recently by estimating the relative importance of input units. Recent research has revealed, however, that such processes tend to misidentify irrelevant input units when explaining them. This is due to the fact that language representation layers are initialized by pre-trained word embedding that is not context-dependent. Such a lack of context-dependent knowledge in the initial layer makes it difficult for the model to concentrate on the important aspects of input. Usually, this does not impact the performance of the model, but the explainability differs from human understanding. Hence, in this paper, we propose an ensemble method to use logic-based information from the Tsetlin Machine to embed it into the initial representation layer in the neural network to enhance the model in terms of explainability. We obtain the global clause score for each word in the vocabulary and feed it into the neural network layer as context-dependent information. Our experiments show that the ensemble method enhances the explainability of the attention layer without sacrificing any performance of the model and even outperforming in some datasets. Full article
(This article belongs to the Special Issue Natural Language Processing with Tsetlin Machines and Deep Neural Networks)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop