Antivirus Evasion Methods in Modern Operating Systems

Round 1

Reviewer 1 Report

As we all know, the security of information system is very important.

The manuscript summarizes and discusses the recent research progress from two aspects of attack and defense.

Overall, the content is clear and logical.

A little improvement suggestion, if possible, the manuscript should include discussions on the relationship between security protection and computing resources, and the balance between security and usability.

Author Response

Thank you very much for review and improvement suggestion. As we focused on modern operating systems and current computers, we assumed that they are powerful enough to not be a consideration for antiviruses software. However, we totally agree that usability vs performance vs security is must-have research topic for different class of devices such as IoT devices and/or personal devices such as mobile phones.  In fact, we are conducting another research about security in different-that-computers devices, where conclusion is security is often omitted due to lack of performance, which in my opinion is not the best architecture. We added short remark about relationship between security, usability and performance and planned works in Future work section.

Reviewer 2 Report

Thank you for submitting your paper. I really like the topic of this work, and believe this work could deliver major contribution to anti-virus research. However, I would like to point out a few problems of this paper, followed by minor comments/questions:

1. The paper claims to be "comprehensive" and "investigate how various techniques can be combined to bypass multiple layers of protection" (last paragraph of Sec 2). However, based on the evaluation, I don't see how you "test combination of multiple antivirus bypassing techniques". Each experiment only tests one tool (with a few configurations). If this evaluation methodology matches your goal, you should clearly point out what combination of multiple anti-virus bypassing techniques is applied in each test. Or maybe you want to combine multiple tools for this purpose. The current evaluation, to me, seems like repeating the prior works (described in Sec 2) with more up-to-date AV software and additional evasion tools (e.g., Hyperion).

2. The technique section (Sec 5) spent too much text on the details of how to use those tools. Most of these details are unnecessary and distractive to the readers. For example, I don't see the significance in including Linux commands in the text. Even if that is necessary, I don't find explanation about those commands. Please consider moving those details into Appendix.

3. The evaluation needs some work. First, each experiment only shows one data point (e.g., Y/N in every tables). Does this mean the result is the same regardless of what payload you generate? If you generate different payloads, is the result still the same? Similar problem occurs when evaluating Shellter (line #419). Have you tried other programs besides ColorPix? Is the result still the same? Likewise, why only picking AV2 in your last experiment (line #464)? 

Second, it is really hard to track your description in the evaluation. For example, in Table 2, you use command-line options (61/62/61-B in Table 2, PP/PPR in Table 3) without explaining what they are. Therefore, it is hard to connect your observations to datapoints in the table. You should replace those with clear text explaining the configuration (e.g., C/powershell, port).

Minor problems:

1. what happens to the line above line #297?

2. you should clearly indicate what Y/N stands for in the tables.

3. move the "point metric" to when you describe Table 5. You didn't use this metric until then so giving that in the beginning is distractive.

4. I don't quite understand your 2nd variant of Hyperion (line #363). If you modify some bytes, how do you recover them to guarantee normal execution? Or you just pick bytes that don't affect program execution?

5. can you confirm that all 6 AV software have version updated between Jan - Sep 2022?

6. what anti-virus evasion mechanism is employed by FatRAT?

7. Please check the reference. The double quote marks are buggy.

Author Response

Thank you very much for review, discussion and valuable improvement suggestion. They will definitely enhance research overall. In accordance to Reviewer’s minor problems, authors corrected paper as follows:

1. Line #297 mismatch was related to misusage of _ sign in that paragraph. This was corrected and the whole paragraph has proper line counting.
2. All tables symbols was explained in Experimental Procedures and Results for Antiviruses Bypass Mechanisms section now. It is related to ‘successful or not’ bypass of antimalware mechanisms.
3. Point metric/Penalty points description was relocated later to be closer to table where we use it for the first time.
4. Authors added commentary on bytes modification. This is one of popular mechanism in shellcode/payload development. Author found hex representation of known command and change it to another equivalent command (opcode). This way, from signature point of view it is a different code, from an execution point of view it does the same operation.
5. Yes, all software was updated (this is the reason why the results was different in a few cases). We added proper explanation/additional comments on this updated in the paper.
6. FatRAT uses a combination of various evasion mechanisms such as obfuscation, encryption, polymorphism. We added proper explanation on evasion mechanisms used by FatRAT in the paper in 5.2.
7. The references were checked, we decided according the suggestion to delete double quote marks to keep reference more clear to read.

In the response to bigger comments/questions:

1. Regarding goal of the paper: Author uses limited, by by far the most popular frameworks used to built current malware by attackers in malicious campaigns nowadays. It is crucial to note that usage of single framework doesn’t imply usage of single technique, f.e. FatRAT creates payload using multiple evasion techniques – signature evasion, obfuscation, encryption etc. In author’s opinion completeness of this research lays inside taking top attackers’ frameworks, best-performing antiviruses and compare effectiveness of defending against 13 different combinations covering more than 90% of malware attacks (excluding zero-day exploits). Goal of this paper was also to be some kind of eye-opener for reader, than simply having antivirus might not be enough in more sophisticated cases and also remind about keeping it up to date. The current evaluation, as stated in rebuilt related work section, covers also running different-than-default parameters (1) of evasion tools and its newer (2) and better versions than it was in prior works – in author’s opinion this is a distinct contribution to state-of-the-art. Of course, I agree with overall commentary and accordingly corrected article in a few places such as related work, antiviruses evasion mechanisms, research environment and toolset and experimental procedures, to make goal of research more clear to reader. Also some of the suggestion of how to enhance research topic was also added to future work section.
2. Regarding details of how to use tool: Author wants to create possibilities to redo tests by another group of researchers for different set of antiviruses (or for different types of antiviruses such as mobile devices avs) and also for future researches with newer version of Avs and/or upgraded version of evasion mechanisms frameworks. However, I agree that some ot the comments were unnecessary and distractive so section 5 was corrected.
3. Regarding evaluation: (1) each evaluation is a single test with Yes/No results regarding detecton by antivirus. From security nomenclature – payload is simply code that will be run, in each test there is the same payload/shellcode used (see Table 5 – first row – basic payload). That means, that we generated payload once – detected by all Avs and during tests, authors modify them using various frameworks and techniques in multiple combinations. Authors clarify this in testing procedure description. (2) similar rule applies to combining payload into another program (Shellter case) – from security point of view, ColorPix is only a transition mechanism – we can choose any not-malicious program and yeld the same results (due to the fact, that Avs detects payload inside the program not, the original program itself). Once again – authors clarify this in article. (3) Last experiment, related to old version of AV2 was chosen, just to enhance one of the goals of the research mentioned above – to remember readers about importance of keeping Antiviruses up-to-date. Results, prove the hypothesis – older version is more likely to be bypassed, as this is not the core hypothesis and the results are not surprising, author decided not to continue with the rest of tools. We assumed the results will be similar – older version will be more vulnerable. (4) Description of options were explained right under the tables in more clear form.

Reviewer 3 Report

Evasion of anti-malware detection is an open research problem and a literature review of recent research is needed. This paper discussed a good number of tools and compared between them in terms of their ability to evade six anti-malware. 

Although the paper did a good job on the experimental part with tools, there is a missing coverage of the research literature, especially Section 4. Each of the subsections should have been given more space. For example, the use of encryption was covered by one paragraph where one reference was discussed. The reference is from 2012. This is problematic as many papers have been published since then. As such, the paper should have covered more research papers on anti-malware evasion in each category mention in Section 4.

Author Response

Thank you very much for review and improvement suggestions. We focused more on using tools that combines different evasion mechanisms, focusing on tool itself, rather than mechanisms underneath.  However, we agree with revision remark to cover more research papers on anti-malware evasion for different categories. In accordance to Reviewer suggestions, we altered the research as follows:

1. We conduct thorough state-of-literature research for different evasion mechanisms for different categories.
2. Each subsection was enhanced with additional up-to-date research presented worldwide.

Reviewer 4 Report

Generally speaking, this manuscript is well organized and well written. The authors demonstrates an in-depth understanding of the research field. However, there are some insufficiencies in the manuscript and should be improved with a minor review.

(1) In the related work, the authors should add some more references to introduce the related research and progress.

(2) There are some grammatical mistakes, the authors should improve them correctly.

(3) It is better for the authors to provide a detailed analyze to explain the advantages of their scheme.

(4) It is better to change the order of section 8 and section 9. That is, in section 8, the authors should introduce the conclusion, then in section 9, the authors should present the future work.

 (5) It is better for the authors to show the Experimental Procedures and Results in figures.

Author Response

Thank you very much for review and improvement suggestions. In accordance to Reviewer suggestions, we altered the research as follows:

1. Related work was enhanced with bigger state-of-knowledge and research as suggested.
2. Whole research was reviewed after all changes and grammatical mistakes corrected, some of sentences rearranged, changed wording to be grammatically correct.
3. Additional commentary was added to Experimental Procedures and Results for Antivirus Bypass Mechanisms to explain why we focused on used scheme – it is related to be comparable to previous research in the state of security of antiviruses and to comply with state-of-art penalty points assessments.
4. As suggested order of sections related to future work and conclusion was changed.
5. Authors added figure related to Experimental Procedures to show attack flow and test architecture as an addition to already written form in research. For results, authors decided to leave the table form due to suggestions from other reviewers to explain used symbols and results. Also in our opinion results presented in table form is better for explanation and clarity.

Round 2

Reviewer 2 Report

Thank you for addressing the comments. One comment: I highly suggest using white background in your figures in your camera-ready version. 

Author Response

Thank you for Your comment. I changed background of figures to white colours.