GaussianMix: Rethinking Receptive Field for Efficient Data Augmentation

Abstract

Mixed Sample Data Augmentation (MSDA) enhances deep learning model generalization by blending a source patch into a target image. Selecting source patches based on image saliency helps to prevent label errors and irrelevant content; however, it relies on computationally expensive saliency detection algorithms. Studies suggest that a convolutional neural network’s receptive field follows a Gaussian distribution, with central pixels being more influential. Leveraging this, we propose GaussianMix, an effective and efficient augmentation strategy that selects source patches using a center-biased Gaussian distribution in order to avoiding additional computational costs. GaussianMix achieves top-1 error rates of 21.26% and 20.09% on ResNet-50 and ResNet-101 for ImageNet classification, respectively, while also improving robustness against adversarial perturbations and enhancing object detection performance.

1. Introduction

Deep Neural Networks (DNNs) have shown promising performance in various aspects of computer vision, including image classification [1,2], object detection [3,4], semantic segmentation [5,6], and more. Modern DNNs typically consist of a significantly large number of parameters. However, with an insufficient number of training samples, these large models may suffer from overfitting problem and poor generalization ability [7,8,9]. Furthermore, the models usually focus on a small region of input images, which results in poor generalization capability on the part of the model. To mitigate such issues, data augmentation strategies have been identified as an effective solution [10,11]. For instance, Cutout [12] achieves a similar effect by removing random regions of the input image. More specifically, Cutout replaces the removed image region with blank pixels; these unwanted image pixels in the augmented image can hurt the model’s performance, as the blank pixels alter the training data distribution [13]. Instead of removing some image regions, CutMix [13] proposes mixing a randomly selected patch (source patch) from another training sample from the target image and also mixing their labels according to the proportion of mixed pixels. Unlike Cutout, this approach prevents the augmented image from having any uninformative image pixels [12]. However, random selection of the source patch may select the background part of the image, which may not correspond to the label of the source image. In order to solve this problem, some methods use saliency information to crop a representative source patch [14,15,16]. Selecting the source patch from around the salient region ensures that the image information is relevant and helps to enhance the model’s performance. However, saliency-based data augmentation methods suffer from two limitations: (i) performance largely depends on the quality of the underlying saliency detection method [17,18,19] and (ii) calculating saliency information introduces computational overhead. To reduce the computational complexity, one potential solution is to compute the saliency information for a single image per mini-batch, then use this information to guide patch selection for all images within that mini-batch. However, this approach may not guarantee selection of the most salient region for each image. Although there are methods for solving a dual optimization problem to ensure selection of salient patches [14], these involve large computational burden. To address the aforementioned challenges, we draw inspiration from the CNN receptive field concept. Studies have shown that the influence of pixels on a CNN’s output follows a Gaussian distribution, with the central pixels contributing more significantly than those at the edges [20]. This suggests that salient information is naturally concentrated around the center of an image. Building on this insight, we propose GaussianMix, a data augmentation strategy that selects salient source patches using a center-biased Gaussian distribution. Notably, unlike traditional saliency-based methods, GaussianMix is independent of the performance of saliency detection algorithms and effectively identifies salient regions without incurring additional computational overhead.

In addition, we performed an experiment to verify the correctness of source patch selection in comparison with SaliencyMix [15], a state-of-the-art method that crops the important source patch using saliency information. To this end, we counted the number of times a pixel was considered to be salient and accumulated the result for all the images in the CIFAR-100 [21] and ImageNet [22] datasets. The resulting heatmaps are presented in Figure 1. In the case of SaliencyMix on CIFAR-100, the result is a Gaussian distribution which seems somewhat tilted. In the case of SaliencyMix on ImageNet, the patches are highly skewed to the left side, which cannot satisfy the receptive field theory [20]. On the other hand, GaussianMix selects the source patches in an improved way, helping to generate diverse samples. To verify the effectiveness of the proposed GaussianMix, we present extensive experiments on various CNN architectures, benchmark datasets, and tasks. To the best of our knowledge, GaussianMix obtains a new best top-1 error rate of 16.28% for WideResNet 28-10 on CIFAR-100 [21]. On the ImageNet classification problem, GaussianMix achieves the best known top-1 and top-5 error rates of 21.24% and 5.83% for ResNet-50 and 19.80% and 4.70% for ResNet-101, respectively. The contributions of this paper are summarized as follows:

• To the best of our knowledge, this work is the first to leverage CNN receptive field theory for the selection of an appropriate source patch for data augmentation.
• Our method selects the salient source patch without relying on a saliency detection algorithm, resulting in significantly improved augmentation efficiency compared to saliency-based and dual-optimization methods.
• The proposed method outperforms state-of-the-art data augmentation methods on image classification, object detection, and adversarial robustness tasks.    

The structure of this paper is as follows: Section 2 reviews related works; Section 3 outlines the proposed method; in Section 4, we evaluate the proposed data augmentation technique through experiments on image classification and object detection, present a time complexity analysis, and assess our method’s robustness against adversarial attacks; Section 5 provides ablation studies on the design choices and hyperparameters along with visualizations of class activation maps [23] for further validation; finally, Section 6 and Section 7 discuss the strengths, limitations, and potential future directions of the proposed approach.

2. Related Work

2.1. Data Augmentation

Due to their large model sizes, a significant amount of data is required to train deep learning models. However, collecting labeled data is a time-consuming and tedious job. To address this problem, various Data Augmentation (DA) methods have been proposed; these apply different transformations on existing data to create new samples with high variation. Traditional DA methods [21,22,24,25] increase the number of samples or their variations by applying simple image transformations, introducing color distortions, injecting noise, Gaussian smoothing, etc. However, traditional DA methods fail to enhance the model’s robustness against occlusion. Several recent studies have suggested mixing different image patches to enhance the generalization ability of CNNs [12,13,14,15,16]. For example, Cutout erases random image regions [12]; however, this can hurt the model’s performance, as the missing pixels alter the training data distribution [13]. Instead, Mixup [26] proposes mixing two training samples and their labels to generate an augmented image. However, the augmented images are locally ambiguous. Considering the limitations of Cutout [12] and Mixup [26], CutMix [13] proposes cutting a patch from a sample and mixing it with another training image. Due to the success of CutMix [13], Mixed Sample-based Data Augmentation (MSDA) techniques have drawn much attention. Because MSDA combines multiple images to generate new samples, it occludes various parts of the object, preventing the network from focusing only on the specific parts that are most discriminative; in other words, it guides the model to consider the less discriminative parts in order to enhance the model’s generalization ability. In MSDA, the augmented image along with the mixed labels confers a label smoothing effect on the network that prevents the model from being overconfident [27]. MSDA techniques can be classified into two categories, namely, random patch-based and salient patch-based.

In random patch-based methods, the source patch is selected in a random fashion and then mixed with the target image. CutMix [13] replaces the removed region of an image with a randomly selected patch from another image (source image) and mixes their labels according to the proportion of mixed pixels. FMix [28] strives to improve upon CutMix by employing arbitrarily-shaped binary masks generated from low-frequency components in Fourier space instead of rectangular patch mixing. Random Image Cropping and Patching (RICAP) [29] cuts and mixes patches from four different training images to create a new augmented image. While these methods are simple and efficient, they do not consider the information contained in the selected patch. This may generate irrelevant augmented data, leading the network to learn unexpected feature representations [15]. To solve this problem, several salient patch-based mixing methods have been introduced.

Saliency-based augmentation techniques select the source patch in such a way that it retains relevant information from the source object. Saliency information emphasizes the significant regions of an image based on the natural attention mechanisms of the Human Visual System (HVS) [14,15,16,30], and is commonly employed for patch selection [14,15,16]. PuzzleMix [14] maximizes the saliency information in the augmented image by solving a dual optimization problem, i.e., it mixes two images so that the augmented image contains as much salient information as possible from the two images. Similarly, Co-mixup [16] combines and matches the collection of salient areas by utilizing inter-arrangements among the mini-batch. SaliencyMix [15] selects the source patch from the salient region of the source image with the help of a saliency detection algorithm and mixes it with the target image. Although these techniques improve model performance, they have certain limitations. Their reliance on saliency detection algorithms means that the effectiveness of the augmentation method is contingent on the quality of the underlying detection process. Additionally, computing saliency information for images imposes significant computational overhead. In contrast, our proposed GaussianMix selects the source patch using a Gaussian distribution that aligns with receptive field properties, where central pixels hold greater importance than those at the edges. By integrating characteristics of the Human Visual System (HVS), our method enables more effective patch selection without relying on a saliency detection algorithm, eliminating the need for additional computational overhead. In this context, it is also worthwhile to mention GridMix [31], which explores spatial modulation for neural fields in the context of PDE modeling. However, we respectfully note that it is fundamentally different from our proposed GaussianMix. While GridMix [31] aims to improve neural field representations in order to solve partial differential equations, GaussianMix is an image augmentation strategy tailored to enhance model generalization for high-level computer vision tasks. Thus, the goals, methodological frameworks, and application domains of the two approaches are distinct.

2.2. Receptive Field

The receptive field of a CNN represents the influence of all image pixels on the network’s output, and is a fundamental concept in convolutional architectures [20,32]. However, not all pixels contribute equally; rather, their impacts vary based on location due to the nature of the convolution operation [20]. Central pixels have multiple pathways by which to propagate information, whereas edge pixels have fewer, leading to center pixels having greater importance for both forward and backward passes. Luo et al. [20] suggested that this influence follows a Gaussian distribution, where central pixels play a more significant role than those at the periphery. Building on this insight, we propose GaussianMix, an effective data augmentation method that selects the source patch using a center-biased Gaussian distribution, thereby leveraging receptive field properties to enhance model performance.

3. Proposed Methods

This section describes the proposed method, which involves the two stages illustrated in Figure 2: first selecting a salient source patch, then incorporating it into the target image.

3.1. Selection of Source Patch

The goal of GaussianMix is to select the source patch from center-biased regions in to order cut a salient part from the source image, then mix it with the target image to generate the augmented sample. Specifically, we select the source patch based on a Gaussian distribution following receptive field theory. Let C denote the center pixel of the source patch; $C_x$ and $C_y$ are the x and y coordinates of the center pixel of the source patch, respectively, and are defined as follows:

$$C_x=N(\mu ,{\sigma }^2)$$

(1)

$$C_y=N(\mu ,{\sigma }^2)$$

(2)

where N is the normal distribution with mean $\mu$ and standard deviation $\sigma$. Following prior works [13,15], the combination ratio $\lambda$ between two samples is randomly determined using a beta distribution $Beta(\alpha ,\alpha )$ in which $\alpha$ is set to 1. The width and height of the patch can be defined as follows:

$$W_p=W\sqrt{1-\lambda }$$

(3)

$$H_p=H\sqrt{1-\lambda }$$

(4)

where $W_p$ and $H_p$ respectively denote the width and height of the selected patch, W is the width of the input image, and H is its height. Figure 2 presents the workflow diagram of the proposed GaussianMix data augmentation.

3.2. Mixing the Patches and Labels

Let $X_A\in R^{W\times H\times C}$ and $X_B\in R^{W\times H\times C}$ be two randomly selected training images with labels $Y_A$ and $Y_B$, respectively. By cropping a patch from $X_B$ and then mixing it with $X_A$, we obtain the augmented image $X^{\prime }\in R^{W\times H\times C}$ as

$$X^{\prime }=M\odot X_A+M^{\prime }\odot X_B,$$

(5)

where $M\in {\{0,1\}}^{W\times H}$ is a binary mask, $M^{\prime }$ is the complement of M, and ⊙ represents the element-wise multiplication. We also mix their labels to get the augmented label $Y^{\prime }$ as

$$Y^{\prime }=\lambda Y_A+(1-\lambda )Y_B\mathrm{s}.\mathrm{t}.\lambda =1-{\displaystyle \frac{W_p{H}_p}{WH}}.$$

(6)

The augmented image $X’$ contains the information of classes $X_A$ and $X_B$, and the augmented label $Y^{\prime }$ also represents this fact, where the weights of the labels depend on the size of the cropped patches as determined by $\lambda$. Algorithm 1 outlines the whole process of selecting and mixing salient source patches in target images of a mini-batch. Additionally, please refer to Section 5.1 for an analysis of different mixing strategies.

Algorithm 1: Applying GaussianMix to a mini-batch

4. Experiments and Results

In this section, we first verify the effectiveness of our proposed method on image classification problem. For fair comparison, we follow previous methods [13,15] in using the CIFAR-100 [21] and ImageNet [22] datasets. We then present a comparison of the computational complexity of our method with other approaches to demonstrate its efficiency. We further perform object detection via transfer learning with a GaussianMix pretrained model, and also investigate the robustness of our proposed method on the adversarially perturbed ImageNet validation set. Finally, we provide an ablation study of different mixing strategies and a class activation map [23] comparison of various methods to further validate the effectiveness of data augmentation with GaussianMix. All experiments were performed using PyTorch 1.13.1 on a system with four NVIDIA GeForce RTX 2080 Ti GPUs and an Intel Xeon Silver 4214 2.20 GHz CPU.

4.1. Classification

We evaluated the performance of GaussianMix data augmentation on both the CIFAR-100 and ImageNet datasets using the following standard deep residual networks: ResNet-18, ResNet-50, and WideResNet-28-10 for CIFAR-100 along with ResNet-50 and ResNet-101 for ImageNet. For CIFAR-100, we trained the models for 200 epochs with a batch size of 128 using Stochastic Gradient Descent (SGD), Nesterov momentum of 0.9, and weight decay of $5\times {10}^{-4}$. The learning rate was initially set to 0.1 and then decreased by a factor of 0.2 at 60, 120, and 160 epochs. We set the $\beta$ hyperparameter to 1.0 and the $\sigma$ hyperparameter to 10.0, as described in Section 5.3. We conducted the experiment three times and report the mean Top-1 error along with the corresponding standard deviation. For ImageNet, we trained the models for 300 epochs with a batch size of 128, using a similar learning rate schedule (initially set to 0.1, then decaying by 0.1 at epochs 75, 150, and 225). The $\sigma$ hyperparameter was set to 40.0, as detailed in Section 5.3. As shown in

Table 1. Classification comparison of various data augmentation methods on CIFAR-100. (Bold depicts best performance).

Method	Top-1 Error
ResNet-18 (Baseline)	22.46 ± 0.3
ResNet-18+CutOut	21.96 ± 0.24
ResNet-18+FMix	20.15 ± 0.27
ResNet-18+CutMix	19.42 ± 0.24
ResNet-18+SaliencyMix	19.29 ± 0.21
ResNet-18+PuzzleMix (200 Epoch)	31.78 ± 0.20
ResNet-18+PuzzleMix (1200 Epoch)	19.66 ± 0.18
ResNet-18+GaussianMix	18.97 ± 0.14
ResNet-50 (BASELINE)	21.58 ± 0.43
ResNet-50+CutOut	21.38 ± 0.69
ResNet-50+CutMix	18.72 ± 0.23
ResNet-50+SaliencyMix	18.57 ± 0.29
ResNet-50+PuzzleMix (200 Epoch)	26.61 ± 0.51
ResNet-50+PuzzleMix (1200 Epoch)	17.17 ± 0.42
ResNet-50+GaussianMix	18.50 ± 0.37
WideResNet28-10 (BASELINE)	18.80 ± 0.08
WideResNet28-10+CutOut	18.41 ± 0.27
WideResNet28-10+FMix	17.97 ± 0.27
WideResNet28-10+CutMix	16.66 ± 0.20
WideResNet28-10+SaliencyMix	16.56 ± 0.17
WideResNet28-10+PuzzleMix	16.23 ± 0.17
WideResNet28-10+GaussianMix	16.28 ± 0.26

, on CIFAR-100, GaussianMix outperforms all other augmentation methods when applied to the ResNet-18 and ResNet-50 architectures, achieving new best top-1 error scores of 18.97% and 18.50%, respectively. (The top-1 error is computed as the percentage of test samples for which the model’s highest-probability prediction does not match the ground truth label, and is equivalent to 100 − top-1 $accuracy(\%)$). For WideResNet, GaussianMix outperforms other state-of-the-art methods such as Cutout [12], CutMix [13], and SaliencyMix [15] while delivering comparable performance to PuzzleMix [14]. Notably, PuzzleMix requires solving a dual-optimization problem that introduces substantial computational overhead and needs 1200 epochs to converge, whereas GaussianMix converges in just 200 epochs. Similarly, on ImageNet, our method outperforms all state-of-the-art methods while achieving comparable performance to PuzzleMix for both ResNet-50 and ResNet-101 as shown in

Table 2. Classification comparison of various data augmentation methods on ImageNet. (Bold depicts best performance).

Method	Top-1 Error
ResNet-50 (Baseline)	23.68
ResNet-50+CutOut	22.93
ResNet-50+Mixup	22.58
ResNet-50+CutMix	21.40
ResNet-50+SaliencyMix	21.26
ResNet-50+PuzzleMix	21.24
ResNet-50+GaussianMix	21.24
ResNet-101 (BASELINE)	21.87
ResNet-101+Cutout	22.30
ResNet-101+Cutout	20.72
ResNet-101+Mixup	20.57
ResNet-101+CutMix	20.17
ResNet-101+SaliencyMix	20.09
ResNet-101+PuzzleMix	19.71
ResNet-101+GaussianMix	19.80

. For ResNet-50, GaussianMix provides a significant improvement in top-1 error of 1.69%, 1.34%, and 0.16% over Cutout [12], Mixup [26], and CutMix [13], respectively. Furthermore, it establishes a new benchmark by achieving a top-1 error of 19.80% on ImageNet when applied to ResNet-101. In addition, the proposed method outperforms SaliencyMix [15] while significantly reducing computational complexity thanks to GaussianMix eliminating the need for saliency detection. The results across both datasets highlight the efficacy of the GaussianMix augmentation strategy, emphasizing its ability to enhance model performance while reducing computational complexity compared to existing state-of-the-art methods.

4.2. Computational Complexity

To demonstrate the computational efficiency of the proposed method, we compared it with various methods in terms of the time required to augment a single image along with the training time. We used WideResNet-28-10 [2] for training on the CIFAR-100 dataset over 200 epochs, and used ImageNet to augment the single images.

Figure 3 shows a comparison of augmentation and training times across different methods. For single images, GaussianMix demonstrates a comparable augmentation time to CutMix [13], while achieving substantial reductions of 45.54% and 80.39% in comparison to SaliencyMix [15] and PuzzleMix [14], respectively. Likewise, GaussianMix requires a training duration similar to CutMix [13] in terms of overall training time complexity, while significantly reducing training time by 97.5% and 99.93% compared to SaliencyMix [15] and PuzzleMix [14], respectively.

4.3. Object Detection

In this section, we examine the effect of the proposed GaussianMix on object detection via transfer learning. We used the Faster R-CNN [3] model for the object detection task. This model originally utilizes VGG-16 [33] as the backbone network. We replaced the VGG-16 backbone with ResNet-50 [1] and initialized it with the GaussianMix pretrained model (trained with ResNet-50). The model was then fine-tuned on Pascal VOC 2007 and 2012 [34] training data and evaluated on Pascal VOC 2007 test data. The performance was measured using the Mean Average Precision (mAP) metric. We followed the fine-tuning strategy of the original method [3]. The batch size, learning rate, and training iterations were set to 8, $4\times {10}^{-3}$, and 41 K, respectively. The learning rate was decayed by a factor of 0.1 at 33 K iterations.

Table 3. Impact of GaussianMix on transfer learning with a pretrained object detection model.

Backbone Network	ImageNet Classification Error	Object Detection Performance
	Top-1 (%)	Faster-RCNN (mAP)
ResNet-101 (Baseline)	21.87	77.53 (+0.00)
CutMix-Trained	20.17	80.04 (+2.51)
SaliencyMix-Trained	20.09	79.91 (+2.38)
GaussianMix-Trained	19.80	80.00 (+2.47)

shows the results. Transfer learning with data augmentation-enabled pretrained models significantly enhances the performance of Faster R-CNN [3], particularly by improving the localization capability of the deep neural network. As precise localization directly contributes to better detection accuracy, leveraging data augmentation during pretraining leads to improved detection performance. Notably, pretraining with our proposed GaussianMix surpasses other data augmentation methods, achieving a performance gain of +2.47 mAP. These results demonstrate the superior effectiveness of GaussianMix in enhancing detection performance.

4.4. Adversarial Robustness

Deep learning models are highly vulnerable to adversarial attacks [35], where small and even imperceptible perturbations can result in misleading predictions [36,37]. Expanding the training dataset through data augmentation improves model robustness by generating diverse samples beyond the original dataset [38]. To assess this improvement, we evaluated various data augmentation techniques using a white-box adversarial attack, namely, the Fast Gradient Sign Method (FGSM) [37], on the ImageNet validation set. To evaluate the robustness of different augmentation methods, we employed a pretrained ResNet-101 [1]. As shown in

Table 4. Comparison of adversarial attack robustness. The table shows the top-1 accuracy of the ResNet-101 architecture trained using various data augmentation methods, with results reported on the ImageNet validation dataset. GaussianMix achieves the highest performance, with a top-1 accuracy of 41.85%. (Bold depicts best performance).

Baseline	CutMix	SaliencyMix	GaussianMix
23.71	41.83	40.49	41.85

, the results indicate that GaussianMix enhances robustness more effectively than SaliencyMix [15] or CutMix [13]. Unlike SaliencyMix, which extracts deterministic patches, GaussianMix incorporates probability-based diverse image regions, leading to broader variation in augmented samples. This results in a performance gain of 1.36% over SaliencyMix and 0.02% over CutMix, demonstrating the superior adversarial robustness of our proposed approach.

It is important to note that while the observed improvement under the tested FGSM attack is modest and not statistically significant as indicated by a z-test conducted with 95% confidence [39] (i.e., $p>0.05$), it was achieved without any degradation in clean accuracy. This outcome suggests that the proposed method contributes to enhanced adversarial robustness while preserving standard classification performance. In adversarial contexts, even marginal gains can be valuable, particularly when clean accuracy remains unaffected. Although these initial findings are encouraging, a comprehensive evaluation across a wider range of adversarial scenarios falls outside the scope of the current work and is left as a direction for future research.

5. Ablation Study

5.1. Mixing Strategies

There are several ways to integrate the source patch with the target image. In this section, we analyze the impact of different mixing strategies on the proposed method. We conducted experiments using GaussianMix with the ResNet-18 [1] architecture on the CIFAR-100 [21] dataset and the ResNet-50 [1] architecture on the Tiny-ImageNet [22] dataset. For both experiments, training proceeded for 200 epochs. We evaluated four mixing strategies: (i) Corresponding position, which refers to the placement of the source patch at the same spatial location within the target image that it originally occupied in the source image; (ii) Center position, where the source patch is placed at the image center; (iii) Random (Uniform) position, where the source patch is placed at a random location; and (iv) Random (Gaussian) position, where the source patch is positioned based on a Gaussian distribution over the target image.

Table 5. Comparative analysis of mixing strategies for the source patch with target image using ResNet-50 on the CIFAR 100 dataset. (Bold depicts best performance).

Method	Top-1 Error
ResNet-18
+ Corresponding position	18.97 ± 0.14
+ Center position	19.87 ± 0.13
+ Random position (Uniform)	19.22 ± 0.18
+ Random position (Gaussian)	19.62 ± 0.32

and

Table 6. Comparative analysis of mixing strategies for the source patch with target image using ResNet-50 on the Tiny-ImageNet dataset. (Bold depicts best performance).

Method	Top-1 Error
ResNet-50
+ Corresponding position	33.47 ± 1.04
+ Center position	34.55 ± 0.46
+ Random position (Uniform)	34.28 ± 0.38
+ Random position (Gaussian)	34.66 ± 0.36

present the results of our comparative analysis of different mixing strategies. Using the Center position for mixing consistently occludes the central region of the target image, which typically contains the most salient information.

This can result in reduced diversity, potentially limiting the regularization effect. In contrast, the Random (Uniform, Gaussian) position strategies introduce greater diversity, which can enhance performance. However, excessive diversity may lead to over-regularization, ultimately degrading model performance. Among the evaluated strategies, Corresponding position mixing demonstrates the best performance, striking a balance between diversity and regularization. Notably, SaliencyMix [15] exhibits a similar trend, with the Corresponding position strategy yielding the best results. Based on these findings, we adopted the Corresponding position as the default mixing strategy for GaussianMix data augmentation.

5.2. Class Activation Map (CAM)

A Class Activation Map [40] highlights the regions of input that a model focuses on. We compared CAMs from ResNet-50 [1] models trained with state-of-the-art data augmentation methods to our proposed GaussianMix on ImageNet [22]. The baseline model included traditional augmentations such as resizing, cropping, flipping, and jittering. We conducted two types of CAM experiments, the first on unmodified images of different classes and the second simulating real-world partial occlusions by using images with occluded salient regions,. From the CAM maps of the original images shown in Figure 4, it is clear that the proposed GaussianMix improves object localization. Moreover, the CAM maps of original and corresponding occluded images in Figure 5 show that GaussianMix helps the model to properly localize occluded objects. In contrast, CutMix [13] generally focuses only on a small portion of the target and fails to address occlusion. While SaliencyMix [15] and PuzzleMix [14] can identify the target well when it is unobstructed, they focus only on specific parts of the occluded object such as the torso or beak. Overall, GaussianMix demonstrates superior localization capability even with occlusions.

5.3. Exploring Optimal Sigma

We conducted comprehensive experiments to find the optimal sigma for each dataset and determine the impact of the Gaussian distribution by evaluating other distributions. As the value of sigma increases, it approaches the behavior of random distribution, making GaussianMix akin to CutMix. Conversely, a smaller sigma tends to extract only the center region. The optimal value of sigma is determined by the dataset and the model; as shown in Figure 6, it is particularly influenced by the dataset. Therefore, we chose the optimal sigma per dataset via grid search. Specifically, we conducted experiments with various values of $\sigma$ (6.0, 8.0, 10.0, 11.0, and 12.0) using ResNet-18, ResNet-50, and WideResNet-28-10 on the CIFAR-100 dataset. As shown in Figure 6a, the results demonstrate that $\sigma =10$ achieves the highest accuracy for this dataset. Given the higher image resolution in the ImageNet dataset, we explored a broader range of $\sigma$ values (10, 20, 30, 40, 50) with ResNet-50. As shown in Figure 6b, $\sigma =40$ was found to yield the optimal accuracy.

5.4. Different Distributions

To assess the impact of different distributions in selecting source patches, we considered three distributions with distinct characteristics: (i) the Gaussian distribution, which has kurtosis similar to a normal distribution and exhibits center bias; (ii) the Gamma distribution, which is less center-biased; and (iii) the Laplace distribution, which has higher kurtosis than a normal distribution. The distribution curves are visualized in Figure 7. We conducted experiments using ResNet-18 [1] as the baseline model on the CIFAR-100 dataset [21]. As shown in

Table 7. Results of using various distributions when selecting the center pixel of the source patch. Experiments were conducted on CIFAR-100 using the ResNet-18 architecture.

Distribution	Top-1 Error
ResNEt-18 (BASELINE)	22.46
ResNet-18+Laplace distribution	20.15
ResNEt-18+Gamma distribution	19.24
ResNEt-18+Gaussian distribution	18.97

, the Gamma distribution tends to select patches at the image boundaries. This may result in the model focusing on specific parts of the image, and can consequently impair its performance. In contrast, the Laplace distribution predominantly selects the image center. This permits the model to learn from various image regions, allowing it to outperform the Gamma distribution. However, the Laplace distribution may not provide as much variation in the selected patches as the Gaussian distribution, which offers patch diversity while also emphasizing salient regions of the image.

6. Discussion

In this paper, we propose a stochastic sampling approach for MSDA techniques using a Gaussian distribution. Based on the concept that central regions tend to have more influence, the proposed method ensures that the patches selected from the image focus on salient areas. Through classification experiments, we observe that GaussianMix provides the model with more diverse previously unseen images, resulting in improved performance and localization capabilities. In turn, the enhanced localization ability boosts the object detection performance of models initialized with the GaussianMix pretrained model. Additionally, as GaussianMix generates a wide range of diverse samples while considering the key image regions, it contributes to greater robustness against adversarial attacks. Although our approach enhances sample efficiency, there remains a need for further optimization of the algorithm. In this work, we used a grid search method to determine the optimal sigma for various networks, including ResNet-18, ResNet-50, and WideResNet. While GaussianMix performs well, there is potential for improvement by adopting more sophisticated algorithms to determine the optimal hyperparameters. Given that CNN models often utilize depthwise and pointwise convolutions, the significance of each pixel in the receptive field may vary, which could lead to slight variations in the Gaussian distribution. Our experiments show that while the optimal sigma does not change significantly, alternative methods could make the process of finding the optimal sigma more efficient. Bayesian optimization [41] or Tree-structured Parzen Estimation (TPE) [42] could be potential approaches for this. Additionally, reinforcement learning techniques could be explored for hyperparameter optimization, including identification of the optimal sigma value.

7. Conclusions

This paper has introduced GaussianMix, a Mixed Sample-based Data Augmentation (MSDA) technique that selects salient patches based on the receptive field theory of CNNs for improved classification and generalization capabilities. GaussianMix enhances model performance by focusing on important image regions without the need for a saliency detection method and without adding computational cost. When applied to ResNet-101, it achieves a top-1 error rate of 19.80% on ImageNet. When applied to ResNet-18 and ResNet-50, it achieves 18.97% and 18.50%, respectively, on CIFAR-100. In object detection, GaussianMix improves Faster-RCNN performance by 2.47% mAP while also boosting robustness against adversarial attacks, demonstrating its effectiveness as a simple and efficient augmentation method.