MDPI - Publisher of Open Access Journals

22 pages, 2251 KB

Open AccessArticle

Deep Reinforcement Learning for Dynamic Twin Automated Stacking Cranes Scheduling Problem

by Xin Jin, Nan Mi, Wen Song and Qiqiang Li

Electronics 2023, 12(15), 3288; https://doi.org/10.3390/electronics12153288 - 31 Jul 2023

Cited by 2 | Viewed by 2874

Effective dynamic scheduling of twin Automated Stacking Cranes (ASCs) is essential for improving the efficiency of automated storage yards. While Deep Reinforcement Learning (DRL) has shown promise in a variety of scheduling problems, the dynamic twin ASCs scheduling problem is challenging owing to its unique attributes, including the dynamic arrival of containers, sequence-dependent setup and potential ASC interference. A novel DRL method is proposed in this paper to minimize the ASC run time and traffic congestion in the yard. Considering the information interference from ineligible containers, dynamic masked self-attention (DMA) is designed to capture the location-related relationship between containers. Additionally, we propose local information complementary attention (LICA) to supplement congestion-related information for decision making. The embeddings grasped by the LICA-DMA neural architecture can effectively represent the system state. Extensive experiments show that the agent can learn high-quality scheduling policies. Compared with rule-based heuristics, the learned policies have significantly better performance with reasonable time costs. The policies also exhibit impressive generalization ability in unseen scenarios with various scales or distributions. Full article

(This article belongs to the Special Issue Deep Perception in Autonomous Driving)

► Show Figures

Figure 1

22 pages, 4740 KB

Open AccessArticle

Noisy Optimization of Dispatching Policy for the Cranes at the Storage Yard in an Automated Container Terminal

by Jeongmin Kim, Ellen J. Hong, Youngjee Yang and Kwang Ryel Ryu

Appl. Sci. 2021, 11(15), 6922; https://doi.org/10.3390/app11156922 - 28 Jul 2021

Cited by 4 | Viewed by 2714

Abstract

In this paper, we claim that the operation schedule of automated stacking cranes (ASC) in the storage yard of automated container terminals can be built effectively and efficiently by using a crane dispatching policy, and propose a noisy optimization algorithm named N-RTS that can derive such a policy efficiently. To select a job for an ASC, our dispatching policy uses a multi-criteria scoring function to calculate the score of each candidate job using a weighted summation of the evaluations in those criteria. As the calculated score depends on the respective weights of these criteria, and thus a different weight vector gives rise to a different best candidate, a weight vector can be deemed as a policy. A good weight vector, or policy, can be found by a simulation-based search where a candidate policy is evaluated through a computationally expensive simulation of applying the policy to some operation scenarios. We may simplify the simulation to save time but at the cost of sacrificing the evaluation accuracy. N-RTS copes with this dilemma by maintaining a good balance between exploration and exploitation. Experimental results show that the policy derived by N-RTS outperforms other ASC scheduling methods. We also conducted additional experiments using some benchmark functions to validate the performance of N-RTS. Full article

► Show Figures

Figure 1

Search Results (2)

Further Information

Guidelines

MDPI Initiatives

Follow MDPI

Saved Queries

Search Filter Reset All

Years

Feature Papers

Subjects

Journals

Article Types

Countries / Regions

Search Results (2)

Further Information

Guidelines

MDPI Initiatives

Follow MDPI