A Study on the Security of Online Judge System Applied Sandbox Technology

Round 1

Reviewer 1 Report

Summary:

The paper presents a code analysis tool to check whether an online judge system’s sandbox is safe to use. The authors conduct a thorough literature review to organize attack methods and create a threat model for an online judge system. They then design code analysis rules and implement a tool to detect potential security risks in the system.

 

Strengths:

- The paper addresses an important issue regarding the security of online judge systems.

- The authors conducted a thoughtful literature review to organize attack methods and present a clear threat model for the system.

- The implementation of the code analysis tool brings practical contributions to improving the security of online judge systems.

 

Weaknesses:

- The paper lacks clear explanations of the technical details of the code analysis rules.

- The evaluation of the tool's performance was not provided.

Readable

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Positive Review Comments:

·         The study addresses an important concern in today's programming courses regarding the security of online judge systems.

·         The abstract clearly highlights the need for avoiding attacks on online judge systems and emphasizes the importance of enhancing their security.

·         The research focuses on studying and organizing various attack methods on online judge systems, which can contribute to a better understanding of the vulnerabilities.

·         The development of a threat model for the online judge system is a crucial step in identifying potential risks and designing effective countermeasures.

·         The implementation of a code analysis tool provides a practical solution for developers to assess the security of the online judge system and address vulnerabilities promptly.

·         The keywords effectively summarize the key aspects of the study, including the use of sandbox technology, online judge systems, security concerns, and static code analysis.

Negative Review Comments:

 

·         The abstract could provide more specific details about the attack methods and the threat model developed for the online judge system.

·         It would be beneficial to include information on the methodology employed for studying and organizing the attack methods.

·         The abstract could mention the potential implications and benefits of enhancing the security of online judge systems, such as fostering trust and reliability among users.

·         More information about the code analysis tool, its features, and its effectiveness in detecting vulnerabilities would enhance the understanding of its practical implementation.

·         It would be helpful to provide examples or case studies illustrating the application of the code analysis tool in identifying and mitigating security risks in online judge systems.

·         Consider expanding the keywords to include additional relevant terms, such as threat modeling, vulnerability assessment, and risk mitigation.

 

Here are some references that can be used:

·         Mustafa, M., Buttar, A.M., Sajja, G.S., Gour, S., Naved, M. and William, P., 2022. Multitask Learning for Security and Privacy in IoV (Internet of Vehicles). Autonomous Vehicles Volume 1: Using Machine Intelligence, pp.217-233.

 

·         Mustafa, M., Alshare, M., Bhargava, D., Neware, R., Singh, B. and Ngulube, P., 2022. Perceived security risk based on moderating factors for blockchain technology applications in cloud storage to achieve secure healthcare systems. Computational and mathematical methods in medicine, 2022.

The level of writing in the provided abstract appears to be intermediate to advanced in English. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

In this paper, the authors organized and reviewed the currently known attack methods for the online judge system, including Denial-of-Service attack methods, TOCTTOU, various covert channels, and cheating methods. Additionally, the threat model of the online judge system was created through the threat model analysis method. Then the security design principles were analyzed according to the created threat model to design code analysis rules and followed the rules to implement a code analysis tool.

I would like to appreciate the efforts made by the authors for their contribution. This paper is well organized and easy to follow. The idea is novel and the results shows tangible benefits.

However, there exists some problems which should be solved before it is considered for publication:

Firstly, the authors should give a more detailed introduction of the motivation (What is the problem, which the method in the paper is needed, why this method is utilized in this security scenario and how does it achieve superior performance compared to traditional methods).

Secondly, the authors should present more background information so that the readers can get a better understanding of the topic. Many new proposed schemes have taken the utilization of security check into consideration using different techniques. The authors should introduce more newly published papers in this field ("TS-ABOS-CMS: time-bounded secure attribute-based online/offline signature with constant message size for IoT systems [J]. Journal of Systems Architecture, 2022, 123: 102388.”) thus demonstrating the novelty of this manuscript.

Thirdly, the authors are encouraged to point out the limitations of this paper and the future directions. This may shed lights for the readers.

At last, kindly do a proofreading to correct the errors in the text.

Author Response

Please see the attachment

Author Response File: Author Response.pdf