Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.1
1.8
Journals
Algorithms
Special Issues
Advancements in Reinforcement Learning Algorithms
share announcement

Advancements in Reinforcement Learning Algorithms

A special issue of Algorithms (ISSN 1999-4893). This special issue belongs to the section "Evolutionary Algorithms and Machine Learning".

Deadline for manuscript submissions: closed (31 January 2024) | Viewed by 22125

Special Issue Editors

Dr. Mehmet Aydin

E-Mail Website
Guest Editor
Computer Science and Creative Technologies, University of the West of England, Bristol BS16 1QY, UK
Interests: metaheuristics; parallel computing; multi-agent systems; planning and scheduling
Special Issues, Collections and Topics in MDPI journals
Dr. Rafet Durgut

E-Mail Website
Guest Editor
Computer Engineering, Bandirma Onyedi Eylul University, Bandirma 10200, Turkey
Interests: metaheuristic optimization; machine learning; adaptive operator selection; reinforcement learning
Dr. Abdur Rakib

E-Mail Website
Guest Editor
Centre for Future Transport and Cities, Coventry University, Coventry CV1 5FB, UK
Interests: logics and formal verification; simulation and model-based testing; automotive systems; multi-agent context-aware systems
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Reinforcement learning (RL), a modern machine learning paradigm, enables an AI-driven system (known as an agent) to learn in an interactive environment via trial and error using feedback from its own actions and experiences. The basic idea behind RL is to train the agent on a reward-and-punishment mechanism. The agent is rewarded for taking correct actions and punished for the wrong ones. In doing so, the agent aims to maximise the appropriate choices while minimising the wrong ones. Here, learning data provides feedback so that the agent can adapt to changing circumstances to fulfil a specific goal. Based on the feedback responses, the agent assesses its performance and responds appropriately. Although RL is not yet widely used in real-world applications, the research on RL has shown promising results.

The most well-known example application domains of RL are self-driving cars, robotics for industrial automation, business strategy planning, trading and finance, aircraft and robot motion control, healthcare, and gaming, among others. In fact, research on RL has expanded in a variety of areas, making it a prominent topic in studies of AI, machine learning, multiagent systems, and data science. RL researchers have developed theories, algorithms, and systems to address problems in the real world that require learning through feedback over time. This Special Issue on the advancements in RL research will offer an overview of the current state-of-the-art techniques, tools, and applications in this area. We are inviting submissions of original work, theory and algorithms of RL, and applications of RL algorithms to real-life problems addressing practically relevant RL issues.

Topics of interest include (but are not limited to):

  • Reinforcement learning;
  • Q-learning;
  • Temporal differences;
  • Markov decision processes;
  • Deep reinforcement learning;
  • Deep Q-network algorithm;
  • Policy optimisation;
  • Policy-based reinforcement learning;
  • Actor–critic RL algorithms;
  • Constrained reinforcement learning;
  • Multiagent reinforcement learning;
  • Collaborative reinforcement learning;
  • Competitive reinforcement learning;
  • Ensemble and distributional reinforcement learning.

Dr. Mehmet Aydin
Dr. Rafet Durgut
Dr. Abdur Rakib
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Algorithms is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (7 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Editorial

Jump to: Research, Review

2 pages, 150 KiB  
Editorial
Why Reinforcement Learning?
by Mehmet Emin Aydin, Rafet Durgut and Abdur Rakib
Algorithms 2024, 17(6), 269; https://doi.org/10.3390/a17060269 - 20 Jun 2024
Viewed by 1074
Abstract
The term Artificial Intelligence (AI) has come to be one of the most frequently expressed keywords around the globe [...] Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)

Research

Jump to: Editorial, Review

25 pages, 731 KiB  
Article
Transfer Reinforcement Learning for Combinatorial Optimization Problems
by Gleice Kelly Barbosa Souza, Samara Oliveira Silva Santos, André Luiz Carvalho Ottoni, Marcos Santos Oliveira, Daniela Carine Ramires Oliveira and Erivelton Geraldo Nepomuceno
Algorithms 2024, 17(2), 87; https://doi.org/10.3390/a17020087 - 18 Feb 2024
Cited by 2 | Viewed by 2436
Abstract
Reinforcement learning is an important technique in various fields, particularly in automated machine learning for reinforcement learning (AutoRL). The integration of transfer learning (TL) with AutoRL in combinatorial optimization is an area that requires further research. This paper employs both AutoRL and TL [...] Read more.
Reinforcement learning is an important technique in various fields, particularly in automated machine learning for reinforcement learning (AutoRL). The integration of transfer learning (TL) with AutoRL in combinatorial optimization is an area that requires further research. This paper employs both AutoRL and TL to effectively tackle combinatorial optimization challenges, specifically the asymmetric traveling salesman problem (ATSP) and the sequential ordering problem (SOP). A statistical analysis was conducted to assess the impact of TL on the aforementioned problems. Furthermore, the Auto_TL_RL algorithm was introduced as a novel contribution, combining the AutoRL and TL methodologies. Empirical findings strongly support the effectiveness of this integration, resulting in solutions that were significantly more efficient than conventional techniques, with an 85.7% improvement in the preliminary analysis results. Additionally, the computational time was reduced in 13 instances (i.e., in 92.8% of the simulated problems). The TL-integrated model outperformed the optimal benchmarks, demonstrating its superior convergence. The Auto_TL_RL algorithm design allows for smooth transitions between the ATSP and SOP domains. In a comprehensive evaluation, Auto_TL_RL significantly outperformed traditional methodologies in 78% of the instances analyzed. Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)
►▼ Show Figures

Figure 1

15 pages, 2787 KiB  
Article
An Intelligent Control Method for Servo Motor Based on Reinforcement Learning
by Depeng Gao, Shuai Wang, Yuwei Yang, Haifei Zhang, Hao Chen, Xiangxiang Mei, Shuxi Chen and Jianlin Qiu
Algorithms 2024, 17(1), 14; https://doi.org/10.3390/a17010014 - 28 Dec 2023
Cited by 2 | Viewed by 2856
Abstract
Servo motors play an important role in automation equipment and have been used in several manufacturing fields. However, the commonly used control methods need their parameters to be set manually, which is rather difficult, and this means that these methods generally cannot adapt [...] Read more.
Servo motors play an important role in automation equipment and have been used in several manufacturing fields. However, the commonly used control methods need their parameters to be set manually, which is rather difficult, and this means that these methods generally cannot adapt to changes in operation conditions. Therefore, in this study, we propose an intelligent control method for a servo motor based on reinforcement learning and that can train an agent to produce a duty cycle according to the servo error between the current state and the target speed or torque. The proposed method can adjust its control strategy online to reduce the servo error caused by a change in operation conditions. We verify its performance on three different servo motors and control tasks. The experimental results show that the proposed method can achieve smaller servo errors than others in most cases. Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)
►▼ Show Figures

Figure 1

20 pages, 1198 KiB  
Article
Reinforcement Learning Derived High-Alpha Aerobatic Manoeuvres for Fixed Wing Operation in Confined Spaces
by Robert Clarke, Liam Fletcher, Sebastian East and Thomas Richardson
Algorithms 2023, 16(8), 384; https://doi.org/10.3390/a16080384 - 10 Aug 2023
Cited by 2 | Viewed by 1517
Abstract
Reinforcement learning has been used on a variety of control tasks for drones, including, in previous work at the University of Bristol, on perching manoeuvres with sweep-wing aircraft. In this paper, a new aircraft model is presented representing flight up to very high [...] Read more.
Reinforcement learning has been used on a variety of control tasks for drones, including, in previous work at the University of Bristol, on perching manoeuvres with sweep-wing aircraft. In this paper, a new aircraft model is presented representing flight up to very high angles of attack where the aerodynamic models are highly nonlinear. The model is employed to develop high-alpha manoeuvres, using reinforcement learning to exploit the nonlinearities at the edge of the flight envelope, enabling fixed-wing operations in tightly confined spaces. Training networks for multiple manoeuvres is also demonstrated. The approach is shown to generate controllers that take full advantage of the aircraft capability. It is suggested that a combination of these neural network-based controllers, together with classical model predictive control, could be used to operate efficiently within the low alpha flight regime and, yet, respond rapidly in confined spaces where high alpha, agile manoeuvres are required. Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)
►▼ Show Figures

Figure 1

24 pages, 3664 KiB  
Article
Iterative Oblique Decision Trees Deliver Explainable RL Models
by Raphael C. Engelhardt, Marc Oedingen, Moritz Lange, Laurenz Wiskott and Wolfgang Konen
Algorithms 2023, 16(6), 282; https://doi.org/10.3390/a16060282 - 31 May 2023
Cited by 3 | Viewed by 2426
Abstract
The demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DTs) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We investigate [...] Read more.
The demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DTs) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We investigate three algorithms for generating training data for axis-parallel and oblique DTs with the help of DRL agents (“oracles”) and evaluate these methods on classic control problems from OpenAI Gym. The results show that one of our newly developed algorithms, the iterative training, outperforms traditional sampling algorithms, resulting in well-performing DTs that often even surpass the oracle from which they were trained. Even higher dimensional problems can be solved with surprisingly shallow DTs. We discuss the advantages and disadvantages of different sampling methods and insights into the decision-making process made possible by the transparent nature of DTs. Our work contributes to the development of not only powerful but also explainable RL agents and highlights the potential of DTs as a simple and effective alternative to complex DRL models. Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)
►▼ Show Figures

Figure 1

27 pages, 431 KiB  
Article
Reinforcement Learning in a New Keynesian Model
by Szabolcs Deák, Paul Levine, Joseph Pearlman and Bo Yang
Algorithms 2023, 16(6), 280; https://doi.org/10.3390/a16060280 - 31 May 2023
Cited by 1 | Viewed by 2139
Abstract
We construct a New Keynesian (NK) behavioural macroeconomic model with bounded-rationality (BR) and heterogeneous agents. We solve and simulate the model using a third-order approximation for a given policy and evaluate its properties using this solution. The model is inhabited by fully rational [...] Read more.
We construct a New Keynesian (NK) behavioural macroeconomic model with bounded-rationality (BR) and heterogeneous agents. We solve and simulate the model using a third-order approximation for a given policy and evaluate its properties using this solution. The model is inhabited by fully rational (RE) and BR agents. The latter are anticipated utility learners, given their beliefs of aggregate states, and they use simple heuristic rules to forecast aggregate variables exogenous to their micro-environment. In the most general form of the model, RE and BR agents learn from their forecasting errors by observing and comparing them with each other, making the composition of the two types endogenous. This reinforcement learning is then at the core of the heterogeneous expectations model and leads to the striking result that increasing the volatility of exogenous shocks, by assisting the learning process, increases the proportion of RE agents and is welfare-increasing. Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)
►▼ Show Figures

Figure 1

Review

Jump to: Editorial, Research

42 pages, 655 KiB  
Review
Inverse Reinforcement Learning as the Algorithmic Basis for Theory of Mind: Current Methods and Open Problems
by Jaime Ruiz-Serra and Michael S. Harré
Algorithms 2023, 16(2), 68; https://doi.org/10.3390/a16020068 - 19 Jan 2023
Cited by 5 | Viewed by 6262
Abstract
Theory of mind (ToM) is the psychological construct by which we model another’s internal mental states. Through ToM, we adjust our own behaviour to best suit a social context, and therefore it is essential to our everyday interactions with others. In adopting an [...] Read more.
Theory of mind (ToM) is the psychological construct by which we model another’s internal mental states. Through ToM, we adjust our own behaviour to best suit a social context, and therefore it is essential to our everyday interactions with others. In adopting an algorithmic (rather than a psychological or neurological) approach to ToM, we gain insights into cognition that will aid us in building more accurate models for the cognitive and behavioural sciences, as well as enable artificial agents to be more proficient in social interactions as they become more embedded in our everyday lives. Inverse reinforcement learning (IRL) is a class of machine learning methods by which to infer the preferences (rewards as a function of state) of a decision maker from its behaviour (trajectories in a Markov decision process). IRL can provide a computational approach for ToM, as recently outlined by Jara-Ettinger, but this will require a better understanding of the relationship between ToM concepts and existing IRL methods at the algorthmic level. Here, we provide a review of prominent IRL algorithms and their formal descriptions, and discuss the applicability of IRL concepts as the algorithmic basis of a ToM in AI. Full article
(This article belongs to the Special Issue Advancements in Reinforcement Learning Algorithms)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-7
Back to TopTop