Low-Carbon Flexible Job Shop Scheduling Problem Based on Deep Reinforcement Learning

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors introduce an end-to-end deep reinforcement learning
framework called LCGRL to address the Job Shop Scheduling Problem. I regard this to be an important problem in both applied mathematics and the general are of sustainability given its wide application in manufacturing. Application of machine learning in these problems is a prescient concern given the current popularity of DRL.

I have no immediate problems with the scientific content, and the results are not controversial, so the editor may decide if it is suitable for the journal given I deem it sufficiently scientifically rigorous.

The quality of English is high, the figures very good, the mathematics clear and overall I would reference this paper as a good introduction to the problem of machine learning applied in manufacturing. The background is sufficient to place the research with respect to the previous literature.

Author Response

Thank you very much for taking the time to review this manuscript.

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The article "Low-carbon Flexible Job-shop Scheduling Problem based on Deep Reinforcement Learning" presents a solution to the energy consumption challenges in manufacturing. While traditional scheduling methods prioritize completion times over energy efficiency, this study introduces the Low-carbon Flexible Job Shop Scheduling problem (LC-FJSP) and proposes a deep reinforcement learning approach. The model, featuring a complex representation based on the Markov Decision Process, demonstrates improved generalization capabilities. However, following areas need to be addressed in order to improve the quality of the research paper.

·         Research contributions have been mentioned explicitly in the Introduction section, but there is no explicit description for the Novelty of the proposed approach. Novelty differs from the research contribution, therefore needed here.

·         The MDP component is integral to any RL process, therefore needs to be presented with in clear and concise manner with maximum visibility for the reader. The MDP in the proposed paper is lost in the text. At the very least, the states, actions, rewards, needs to be bold in writing for visibility. Secondly, the definition of these is not clarifying the whole picture, so its better to add examples, like if action comprises of process selection and machine assignment, give example by combining a process selection option with the machine assignment option.

·         In the MDP you mentioned the Reward part. Was any negative rewarding policy was considered in the scope of this research? If not, what was the reason for not considering it, i.e not suitable for the scenario, not seen as much fruitful by the authors, etc

·         The authors have deployed the LeakyReLU activation function in the operation feature attention module. What was the reason behind using LeakyReLU instead of ReLU activation function, was the data too noisy or abnormal amount of outliers, or was there something else which wasn’t possible with ReLU?

·         I am unable to see any software or hardware specifications in the experimentation section. These software and hardware specification should be clearly mentioned for the ease of reproducibility goal for the readers. For instance the frameworks and libraries used to build these networks such as PyTorch, Tensorflow, Chain etc along with their versions needs to be mentioned.

·         Does this approach have not any limitations? If yes, then they need to mentioned for instance in the conclusion section, where remedy if known can be proposed as future work if intended.

·         In overall, incorporating multi-head attention modules and Bayesian optimization increases the complexity of the proposed solution, will this complexity increase hinder its implementation and scalability in the real-world manufacturing environments? And when we say manufacturing environments, we mean across different manufacturing contexts or industries, so does the proposed approach the generalizability concern in some manner? Authors need to add some discussion regarding this.

 

·         Are there any potential trade-offs or side effects for this optimization such as increased production costs or any other impacts on any other sustainability metrics? 

Comments on the Quality of English Language

Minor corrections are required during the proofreading process.

Author Response

Thank you very much for taking the time to review this manuscript.We have carefully considered all the suggestions and have made the necessary revisions to improve the quality of our paper. Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Journal: Sustainability (ISSN 2071-1050)

Manuscript ID: sustainability-2999267

Type: Article

Title: Low-carbon Flexible Job-shop Scheduling Problem based on Deep Reinforcement Learning

 

Reviewer’s Comments

Summary:

The paper addresses a significant and timely issue of integrating low-carbon strategies into flexible job shop scheduling (FJSP) using advanced deep reinforcement learning (DRL) techniques. This is an important contribution to both the fields of sustainable manufacturing and machine learning.

The manuscript is well-organized and follows a logical structure, starting from problem definition, methodology, algorithm design, and concluding with experimental validation and discussion. The sections are clearly delineated, making it easy to follow the research flow. However, there are some comments that authors need to update.

Q1: In abstract:

Consider including a specific numerical result in the abstract to highlight the effectiveness of your approach. For example, mention the percentage improvement in scheduling efficiency or reduction in carbon emissions compared to traditional methods.

Q2:  In introduction:

It would be beneficial to include a brief mention of the practical implications and potential industrial applications of the proposed model to emphasize its real-world relevance.

Q3: I would recommend creating a new section of related works to include:

a.     Traditional methods,

b.     Quantum computing,

c.     DRL,

d.     come to support your gap of knowledge.

e.     Ensure that all references are current and relevant. Some references might be updated with more recent studies to reflect the latest advancements in the field.

and try to include.

1)    It would benefit from a discussion on quantum computing-based optimization methods for job shop scheduling. Recent studies utilizing quantum annealing or digital annealing for similar optimization problems could provide valuable insights and a comparative background for your approach. The choice of deep reinforcement learning (DRL) and the introduction of the Low-carbon Graph Attention Network (LCGAN) are well-justified. However, it would be beneficial to explain why alternative optimization methods, such as quantum annealing or digital annealing, were not considered or were deemed less suitable for this problem:

a.     Including a comparative analysis or theoretical discussion on the potential benefits and limitations of quantum computing approaches versus DRL in this context would provide a more comprehensive understanding of your methodological choices.

Q4: In Problem Description and Model Construction:

The problem formulation and mathematical modelling are detailed and clear. The use of a disjunctive graph model and the Markov Decision Process (MDP) for problem representation is well-explained. Consider adding more examples or visual aids to illustrate complex concepts, such as the disjunctive graph model and the operation of the graph attention network. For example,

 

Disjunctive Graph Model: Figure 1 and 2

Create a visual representation of the disjunctive graph model used in the study. This could include nodes representing operations and edges representing the possible sequences of operations on different machines. And highlight the nodes and edges that demonstrate the low-carbon scheduling aspect.

For example, in [Figure: Disjunctive Graph Model]

- Nodes: Represent operations.

- Directed Arcs: Show precedence constraints.

- Undirected Arcs: Represent potential sequences of operations on machines.

Q5: Experiments:

The experimental settings are well-detailed, but the inclusion of more varied datasets, especially real-world datasets with diverse characteristics, would strengthen the validation of your approach. However, it should consider adding a section that discusses the potential scalability of your approach. How does the performance of your algorithm change with an increasing number of jobs and machines? Including such an analysis would highlight the robustness of your method.

Q6: Results:

The results are comprehensively presented, comparing the proposed method with existing benchmarks and heuristic approaches. The inclusion of both synthetic and real-world datasets strengthens the validation. Consider adding a more detailed analysis of the results, including statistical significance tests or confidence intervals, to support the claims of improvement.

Q7: Conclusion:

The conclusion summarizes the key contributions and findings of the study. It also outlines future research directions, which is commendable. Consider explicitly stating any limitations of the current study to provide a balanced view and guide future research efforts.

Q8: Technical and Minor Comments:

·       Ensure all figures and tables are clearly labelled and referenced in the text.

·       Check for any grammatical or typographical errors to improve the readability of the manuscript.

·       Ensure consistency in the use of terms and notation throughout the paper.

·       Make sure to numbering the Equations.

Author Response

Thank you very much for taking the time to review this manuscript.We have carefully considered all the suggestions and have made the necessary revisions to improve the quality of our paper. Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

All the comments have been addressed.