Empirical Perturbation Analysis of Two Adversarial Attacks: Black Box versus White Box

Round 1

Reviewer 1 Report

The paper analyzes the adversarial images created by two completely opposite attacks against 10 ImageNet-trained CNNs. A total of 2 × 437 adversarial images are created by, a black-box evolutionary algorithm, and by BIM, a white-box, gradient-based attack. The authors inspect and compare these two sets of adversarial images through different perspectives: behaviour while looking at smaller regions, noise frequency, transferability, texture 8 change, and penultimate layer activations.

The manuscript has a few editing errors that must be revised, such as the size of the figures, the caption of tables, and the description of the equations as equations using (X) to identify each one. 

The English must be revised. E.g., "behaviour" must be "behavior"

Line 71. Where the authors inform how the paper is organized, the sentences are long. It used short only on paragrapher.

What informations were considered to choose the 10 CNN's presented in this paper? 

Information in table 1 used small-size text. Also, figures 1, and 2 are small.  Check the other figures. 

In some cases in the table, the caption is presented under and other cases over of table. Try to use it the same way. 

In general, the paper has based comparison and analysis of methods of CNNs for a specific problem. It is suggested to provide a new method or technique that can be better the available methods. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

# General summary

The authors investigate perturbed images of a black-box and a white-box attack 

against 10 different ImageNet-trained CNNs. While one of the attack is based on 

an evolutionary algorithm, the other is gradient-based. The paper gives a more 

profound understanding on what happens when a network is attacked adversarial.

 

While it is generally well written and addresses an interesting topic, 

there are some flaws. The main problem, is a complete lack of considering 

related work apart from a short paragraph in section 1. In addition, the 

following more formal issues should also be resolved, before the paper can be

accepted.

 

 

# Figures and Tables

- Please be consistent with the positioning of captions. Some are above 

  (Table 1, Table 3, Table 4) while the rest are beneath the floating object.

- Tables and figures should also be placed either at the beginning or the end 

  of a page.

- Use multicolumn to make the content of Table 4 to Table 7 more readable. The 

  (<EA Value>, <BIM Value>) is quite hard to read.

- Algorithm 1 should be part of the appendix.

 

# Language and Grammar

- Reread the text carefully. Especially the first few pages and the conclusion 

  contain some sentences, that should be rewritten to make them more clear. 

- Check for missing or wrong commas, grammar, capitalization and remove wordy 

  or unnecessary phrases.

- Why are the abbreviations for BIM and EA not written out, but CNN is?

- Fix missing words, e.g.:

  - Line 25 ("Given a trained CNN and a(!n) ancestor image(!,) A classified by 

    <?> as belonging to the category [...]")

  - Line 111 ("2. Adversarial images created by BIM and <?>")

  - Line 116 ("[...] we only give a brief overview of the two algorithms used 

    here, namely <?> and BIM [...]")

  - Line 123 ("[...] which is adversarial for <?> in [...] 

  -...

 

 # Style

 - Some paragraphs (e.g. 2 and 4.4) are indented, some are not. Be consistent 

   and follow the guidelines.

 - Please use empty lines only if they are necessary.

 - There is also a widow and orphan that should be fixed (Line 574-576)

 - Related work is missing

 - 2.3.1 has an empty headline

 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

 

Author Response

There were no comments from Reviewer 1 in the second round. 

Reviewer 2 Report

Thanks for addressing the issues. I have only a few minor things to complain:
The dots at the end of the headlines 2.3.1 and 2.3.2. and some issues with the tables:
- The subcaptions in table 3 and table 7 can be removed if they provide no further information despite c_a, c_t.

- Table 5 and 6 have the column header "For c_a", "For c_t" while Table 7 is missing this header. Is this header correct, since the entries below contain the name of the used networks?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf