Optimizing Maintenance of Energy Supply Systems in City Logistics with Heuristics and Reinforcement Learning

In urban logistics, effective maintenance is crucial for maintaining the reliability and efficiency of energy supply systems, impacting both asset performance and operational stability. This paper addresses the scheduling and routing plans for maintenance of power generation assets over a multi-period horizon. We model this problem as a multi-period team orienteering problem. To address this multi-period challenge, we propose a dual approach: a novel reinforcement learning (RL) framework and a biased-randomized heuristic algorithm. The RL-based method dynamically learns from real-time operational data and evolving asset conditions, adapting to changes in asset health and failure probabilities to optimize decision making. In addition, we develop and apply a biased-randomized heuristic algorithm designed to provide effective solutions within practical computational limits. Our approach is validated through a series of computational experiments comparing the RL model and the heuristic algorithm. The results demonstrate that, when properly trained, the RL-based model is able to offer equivalent or even superior performance compared to the heuristic algorithm.