A Novel Static Analysis Approach Using System Calls for Linux IoT Malware Detection

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

In the proposed paper, the authors suggest a system for statically analyzing Linux ELF binaries for ARM (presumably because most IoT devices are based on this architecture) to detect malware. Specifically, the authors used a larger number of known malware, as well as a similar number of benign software for comparison, extracted syscalls (both actual syscalls as well as library-wrapped ones) and essentially fed these results into different machine learning algorithms in order to arrive at conclusions.

After syscall extraction, the calls are categorized whereas every Linux system call has been assigned to one of ten categories (I presume the categories have been set by the authors themselves, since I did not find any other reference). As for data augmentation, the dataset has been enriched with additional metadata like call count, category frequency, etc.

Overall, both research design and methodology look good, as does the actual execution. I would also like to positively comment on the fact, that the authors provided a Github repo with a ZIP containing the essential data they have worked with. The results look promising, analayis is thorough and properly done.

Very few issues remain:
 - Fig. 1 is of bad quality
 - Fig. 3+4: Neither the diagram nor the subtext tells the reader which data is supposed to come from benign or malign binaries; differentiating between "binary data 1/0" is confusing - rather provide a proper distinction like in Fig. 5.
 - related work: the text itself is fine, but I find it irritating that the authors only included other research which tried to focus on dynamic analysis, rather than static approaches like presented in this proposal
 - Even though the authors write that their approach does not intend to replace dynamic analysis, the general consensus is that due to the major limitations of static analysis, such schemes tend to perform worse in certain situations - e.g. when malware authors are using AV evasion techniques, encryption, etc. It would have been nice to see a few comments on that to draw a clearer picture.

All in all, this is a properly executed research paper.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Overall Impression

_{The paper, "A Novel Static Analysis Approach Using System Calls for Linux IoT Malware Detection," presents a comprehensive study on the use of static analysis to detect malware in Linux IoT systems. The approach of using system calls and library syscall wrappers extracted through static analysis to identify malicious behaviour is innovative and promising. The methodology is well-structured, and the results demonstrate the efficacy of the proposed method with a high F1 score of 96.86%}

Strengths:

_{1 - Innovative Approach: The paper introduces a novel approach by focusing on static 2 analysis for syscall extraction, which reduces the risks associated with dynamic analysis.}

_{2 - Comprehensive Methodology: The authors provide a detailed explanation of the workflow, from dataset creation to machine learning model evaluation, ensuring reproducibility.}

_{3 - High Accuracy: The results show strong performance across multiple machine learning models, indicating the robustness of the proposed method.}

_{4 - Detailed Statistical Analysis: The inclusion of statistical tests, such as the chi-squared and Mann-Whitney U tests, to validate the significance of syscall categorization and ranking adds credibility to the findings.}

Weaknesses:

_{1 - Scope of Analysis: The study focuses primarily on syscalls, potentially missing other significant static features that could improve detection rates.}

_{2 - Dynamic Behaviour: Static analysis cannot capture dynamically generated syscalls, limiting the understanding of full malware behaviour.}

_{3 - Figures and Tables requires more readability.}

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The paper, entitled "A Novel Static Analysis Approach Using System Calls for LinuxIoT Malware Detection," is presented at a high level of professional quality. The paper addresses an intriguing subject and presents novel and innovative findings.

The paper demonstrates potential for further development, particularly in the following areas:

 In the area of resources, the section mapping the current state of knowledge must be expanded to include a more comprehensive view.

In the methodology section, the methods presented must be backed up by appropriate and relevant sources. This will ensure that the reader is aware of the basis for the methods and the procedures they develop and apply. It would be beneficial to extend the datasets with additional distributions, whether to highlight relevant differences or to identify matches.

The presentation of results could be improved by making them more readable, especially in the form of graphs.

 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The questions addressed in the research are the feasibility and effectiveness of using static analysis, specifically examining system calls (syscalls) and library syscall wrappers, for detecting Linux IoT malware. The study aims to establish whether static analysis can reliably differentiate between malicious and benign binaries without the need for dynamic execution. The paper addresses a significant gap in the field where most existing studies rely on dynamic analysis, which involves executing the malware to understand its behavior. By focusing on static analysis, this paper provides a potentially safer and more efficient method for early malware detection in IoT devices. The conclusion is well written with evidence and arguments presented. The study thoroughly validates the effectiveness of static analysis for syscall extraction and malware detection through statistical tests and machine learning evaluations. 

The dataset, while extensive, is limited to ARM architecture, potentially restricting the generalizability of the findings. Additionally, the reliance on statistical validation without real-world deployment scenarios raises questions about practical applicability. The paper does discuss limitations, but further elaboration on potential challenges in scaling the approach to different IoT devices and environments could be beneficial. Finally, while the study demonstrates high accuracy, the absence of detailed error analysis leaves room for skepticism regarding the robustness of the results under different conditions.

 

 

Author Response

Please see the attachment

Author Response File: Author Response.pdf