A Deep Reinforcement Learning Floorplanning Algorithm Based on Sequence Pairsâ€
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis paper introduces a deep reinforcement learning algorithm based on sequence pairs to optimize VLSI floorplanning, demonstrating superior results on standard benchmarks. Here some comments:
It is recommended that the authors add more references about the use of RL in the floorplanning problem, what gaps in knowledge exist and how it relates to the proposed method.
Authors are recommended to highlight the main contributions of the paper in the introduction.
How do the authors avoid that the definition of the function to be maximized does not lead to a suboptimal policy of actions?
it would be beneficial to address in the conclusion the potential future directions or improvements for the proposed algorithm and to briefly discuss its applicability to real-world VLSI design scenarios.
Comments on the Quality of English LanguageMinor editing of English language required
Author Response
Please see the attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsA Deep Reinforcement Learning Floorplanning Algorithm Based on Sequence Pairs
Approved, after this minor suggestions:
While Moore's Law is referenced, it would be beneficial to include citations for other claims and statements, especially when mentioning previous algorithms or categorizing research approaches. This adds credibility and allows readers to explore relevant literature.
Consider structuring the introduction around key themes or objectives, such as the importance of floorplanning, the challenges it poses, previous approaches, and the gap that the proposed algorithm aims to address.
Provide specific quantitative results from the experiments to support the claims of superior performance compared to other algorithms. This could include numerical comparisons of chip area utilization, wirelength, and any other relevant metrics
Conclude the conclusion by discussing potential future research directions or extensions of the proposed algorithm. This could include exploring different reward functions, optimizing hyperparameters, or applying the algorithm to other domains beyond integrated circuit design.
Comments for author File: Comments.pdf
Minor editing of English language required
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe addressed topic is relevant to the area. Although the contributions are interesting, there are major issues with this manuscript that should be addressed.
1) The sentence “Currently, sequence pairs representation is widely used because any floorplan can be represented using sequence pairs, and the required storage space is very small” is quite poor and does not provide a clear understanding on the sequence pairs representation relevance. The author should consider rephrasing this sentence and include one or two references.
2) The related works lack a description on the floorpan representation utilized as well as the limitations of previous studies. Moreover, the Introduction section could benefit from the inclusion of a summary of the work contributions.
3) The reward function presented in Equation (7) is problematic because it seems that the reward is not strictly Markovian as it depends on the decisions made in previous states. Therefore, the addressed problem cannot be defined under the MDP framework. In non-Markovian reward decision processes (NMRDPs), rewards depend on the preceding trajectory, which is exactly the reward utilized in this manuscript. A typical way to address NMRDP problems is by augmenting the state with information that makes the new
model Markovian. Please refer to 10.1609/aaai.v34i04.5814, 10.1609/socs.v8i1.18421, dl.acm.org/doi/abs/10.5555/1867406.1867424, and Bacchus, Fahiem, Craig Boutilier, and Adam Grove. "Rewarding behaviors." Proceedings of the National Conference on Artificial Intelligence. 1996.
4) The experimental section lacks a description about the experimental configuration (ie number of iterations, episode length, etc) and parameters used both in the network of the proposed approach and the compared algorithms.
5) Tables 3 and 4 are missing the column with wire length performance of “Reference [33]”. Moreover, the performance of algorithm from “Reference [33]” is also missing in Tables 5 and 6.
6) The paper needs not only positive statements of the claimed superiority of the proposed method, but a clear statement of its weaknesses.
7) The conclusions are not well summarized. The authors just list the main work, but the "conclusions" are not given.
Minor comments:
Page 3: The example sequence pair “<4, 3, 1, 6, 2, 5” should be “<4, 3, 1, 6, 2, 5>”
Page 6: Equation (7) is missing the parameters in the reward function.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
