RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets

Round 1

Reviewer 1 Report

Summary:
An interesting and clearly written paper describing a new technique for data augmentation to improve the perfmorance of a machine-learning based classification approach applied to radar data. To avoid overfitting, a relatively small dataset is extended by artificial data taking into account effects such as the attenuation and deterioration of lateral resolution with range and changes in the noise floor. Experimental results in a laboratory environment demonstrate the efficiency of the proposed approach. Looking forward to results in a real environment!

Comments:
- Can you summarize the used the standard data augmentation (SDA) in a few sentences? This will make this paper much more approachable.
- Based on the radar images in Fig. 9, there doesn't seem to be any serious interference between the target, as the used radar has very good range and angular resolution and the targets are sufficiently spaced apart. Why are the results in Table 6 so much worse than in Table 5?

Minor problems:
- According to the SI norm, there should be a space between the number and the unit and the unit is not italicized (see 7.2 in https://physics.nist.gov/cuu/pdf/sp811.pdf). So 35cm in line 40 is wrong, 20 GHz in line 46 is correct. Please fix this and make it consistent throughout the paper.
- Line 82: "We used the center of the object to compute X" -> X is not defined.
- Line 93: "Hence, to account for the effect of changing polar resolution with range" -> The polar resolution stays constant with range, the Cartesian resolution deteriorates with range?
- Line 177: Can you give some details which type of CFAR you used (e.g. GO / LO CA, OS)?
- Line 215: Typo "A one" -> "As one".
- Line 217: "results with radar (in gray)" -> "results with RADIO (in gray)"?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper proposes several data augmentation strategies for radar images. The augmentations seem to signficantly improve the accuracy of neural networks for classification and detection.

The paper is well-written and easy to follow.

The experiments are small scale, but well-designed.

In Eq (2), would the different points not also have a different range R, in addition to different RCS sigma?

"From Eq. 2, there is a linear relationship between the log of the received power and range". The range appears in Eq (2) as 1/R^4. So this is not a linear relationship with the log of P_r. That would also mean that augmentation should be P_r^-4 = aR+b, although this wouldn't affect things by much, and perhaps a log-linear relation better fits the empirical attenuation.

In table 2, there still a difference between the real and augmented images at 3.8m, which is presumably due to other agumentations than the range augmentation. Is that understanding correct? Or is it because an independent sample was used? To be able to tell if the range augmentation is comparible to the ground truth it would be useful to also see MSAD for 6.8m without range augmentation, and perhaps also a 3.8m image generated from a 6.8m one.

In section 5.1 it is said that standard RGB augmentation is used, but radar images have only a single channel. How were the radar images treated? Does this agumentation then refer to those from [22] that do not affect color such as affine transformation and Gaussian noise?

Other suggestions:

• Label the images in figure 6.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

This manuscript presents a novel method for augmenting limited radar datasets to train convolutional neural network classifiers. The method is tested on real, lab collected datasets at 300GHz, with 6 different classes of objects of relevance to automotive applications. The results show significant improvements to classification performance when the augmented datasets are used to train the CNNs, albeit for a very specific CNN architecture. The physically-motivated augmentation uses very simple (almost crude) models to generate additional data samples yet is sufficient to extract meaningful performance improvements. The material is presented with sound logical flow and the paper is easy to follow. The claimed contributions are suitably modest and commensurate with the amount of experimental results offered. Overall, the recommendation is to accept this manuscript with some changes.

Specific suggestions:

- Section 2: it is assumed that matched filtering is used to create the range profiles. The fact that FFT is mentioned suggests a stepped frequency waveform? A little more details on the radar and range processing steps would clear up any potential confusion.

- Eq (2) assumes the ranges of the different scattering centres to be the same? This is not likely to be the case with extended targets such as bicycles and trolleys, especially with the bulk target ranges ~10m and a range resolution of 0.75cm.

- Table 2: MSAD is fairly meaningless across different objects – a relative error would be more meaningful. It is surprising that the MSAD at 6.3m are significantly lower than at 3.8m, given real data at 3.8m is used to generate the `fake’ data. Any comments on this surprising result?

- Section 5: a brief description of the `camera based’ data augmentation would be valuable. The cited reference [22] describes a general technique, so its application to the current context merits some discussion.

- Table 5: the number of real samples is 475 in the text, yet listed as 450 in this table?

- Tables 6 & 7 present a large volume of results. What do the N/A mean? It is assumed that no relevant data were collected for these cases. The very poor AP for cone targets are not discussed. Is it purely a consequence of its small size? It is a common object on the road and thus of great relevance to automotive radar scenarios.

- Any comments on the performance as amount of augmented data is varied? A ratio of 41:1 (augmented:real data) is used here. How would things change if this is varied across an order of magnitude either side?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf