Applying Deep Generative Neural Networks to Data Augmentation for Consumer Survey Data with a Small Sample Size

Abstract

Questionnaire consumer survey research is primarily used for marketing research. To obtain credible results, collecting responses from numerous participants is necessary. However, two crucial challenges prevent marketers from conducting large-sample size surveys. The first is cost, as organizations with limited marketing budgets struggle to gather sufficient data. The second involves rare population groups, where it is difficult to obtain representative samples. Furthermore, the increasing awareness of privacy and security concerns has made it challenging to ask sensitive and personal questions, further complicating respondent recruitment. To address these challenges, we augmented small-sized datawith synthesized data generated using deep generative neural networks (DGNNs). The synthesized data from three types of DGNNs (CTGAN, TVAE, and CopulaGAN) were based on seed data. For validation, 11 datasets were prepared: real data (original and seed), synthesized data (CTGAN, TVAE, and CopulaGAN), and augmented data (original + CTGAN, original + TVAE, original + CopulaGAN, seed + CTGAN, seed + TVAE, and seed + CopulaGAN). The large-sample-sized data, termed “original data”, served as the benchmark, whereas the small-sample-sized data acted as the foundation for synthesizing additional data. These datasets were evaluated using machine learning algorithms, particularly focusing on classification tasks. Conclusively, augmenting and synthesizing consumer survey data have shown potential in enhancing predictive performance, irrespective of the dataset’s size. Nonetheless, the challenge remains to minimize discrepancies between the original data and other datasets concerning the values and orders of feature importance. Although the efficacy of all three approaches should be improved in future work, CopulaGAN more accurately grasps the dependencies between the variables in table data compared with the other two DGNNs. The results provide cues for augmenting data with dependencies between variables in various fields.

1. Introduction

Questionnaire survey research is one of the most significant approaches utilized in marketing research. Survey research methods are generally carried out online or offline. Respondents rate questionnaires with five- or seven-point Likert scales for products, advertisements, and their demographic and psychographic information. According to European Society for Opinion and Marketing Research (ESOMAR) [1], quantitative methods, including the questionnaire survey research, are major marketing research approaches that account for more than 60% of the marketing research methods used for sales volume. Although survey research methods play a crucial role in marketing management and represent the prevailing approach toward gathering consumer opinions, several significant issues related to these methods must be resolved. The first problem is the cost issue. The larger the sample size, the closer the acquired data from the survey are to the population. However, the cost of executing survey research methods increases according to the sample size. This suggests that enterprises with a small marketing research budget cannot obtain accurate results from survey research. In many cases, small-size enterprises (e.g., start-up companies and those placed in local areas) cannot afford to invest sufficient budget for executing survey research with a large sample size [2]. As a result, these companies struggle to achieve consumer insights when developing new products and services. They are eventually obliged to execute marketing initiatives restricted to their closed knowledge. Previous studies on small business firms have pointed out that lacking a marketing attitude leads to business failure [3,4]. This suggests that the marketing initiatives of these businesses might be equal to gambling without considering the consumer demands. This marketing research issue in small business firms has remained unsolved for quite a long time now. The second issue concerns rare populations [5,6]. Many cases have targeted limited markets in marketing strategies. These include specialists in particular industry areas (i.e., surgeons, artists, and craftsmen), class groups (i.e., businesspeople with high salaries), and people with attributes (i.e., intractable disease patients and luxurious brand customers). Executing survey research for these rare populations is challenging. This issue is hard to solve in terms of budgeting alone and remains unresolved in the marketing research area. The third issue is the inconvenience of analyzing small-sample-sized data using statistical models, such as predictive models and some statistical testing methods. When predicting outcomes using customer data, several predictive models with small sample sizes could yield low-accuracy results. When performing structural equation model analysis with small-sample-sized data, several parameters in the model might not be calculated. Driver analysis is a useful analytical approach for making marketing decisions based on consumer survey research. Machine learning models, such as linear discriminant model analysis and regression models, can be employed for driver analysis to obtain effective variables for purchasing and preferring products and brands. The results of using small-sample-sized data for driver analysis may be inconsistent with those for a large-sample size. Therefore, the results of analyses using small-sample-sized data are irrelevant when making marketing decisions.

Several techniques for augmenting small-sample-sized data have been presented to overcome the small sample size issue in the image-processing field. The data augmentation techniques are classified into two categories [7]. The first category comprises traditional data augmentation techniques such as cropping, flipping, noise transformation, and random erasing. This approach creates various patterns of seed data using parts and edits them according to certain rules. The Mixup is a more sophisticated technique than traditional techniques [8]. The Mixup is an approach that uses probabilistic distributions based on existing data. It involves randomly selecting data pairs from the existing dataset. In the Mixup, sampling is executed according to the beta distribution. Fast auto augment (Fast AA) and population-based augmentation (PBA) also approach sampling data based on existing data according to augmentation policies [9,10]. The second category comprises deep learning-based data augmentation techniques, such as transfer learning [11] and synthesized data generated by deep generative neural networks (DGNNs) (e.g., the generative adversarial network (GAN) [12,13] and variational autoencoder (VAE) network [14]). Aside from effectively augmenting data, data augmentation methods based on synthesized data generated by DGNNs are effective for security applications [15,16]. The security issue is significant in the analysis of survey research on small sample sizes, including rare populations. Respondents in a survey search are limited, and the characteristics of their population might be easily speculated; hence, invaders can easily specify their individual information. Thus, data augmentation approaches using synthesized data by DGNNs are promising approaches. Therefore, in this study, we focus on a data augmentation technique using synthesized data generated by DGNNs.

The effectiveness of data augmentation techniques using synthesized data generated by DGNNs has been confirmed for a wide variety of computer vision tasks [7,11]. These techniques have been modified and extensively applied to tabular data forms. When addressing a small-sample-sized data issue, such as an imbalanced data problem, oversampling methods are usually used to augment the small data. The synthetic minority oversampling technique (SMOTE) is a primary helpful approach for oversampling small sample-sized data in tabular data, and modifications have been developed (i.e., adaptive synthetic sampling and borderline SMOTE) [17]. This approach’s base method augments minority class data by interpolating data based on the distance between minority and majority data. The usefulness of these oversampling approaches has prevailed in various areas, such as business and physiological data [18,19]. Although these oversampling methods play a crucial role in solving small-sample-sized data problems, we focus on DGNN-based augmentation techniques as data augmentation methods for tabular data, not to address an imbalanced data problem.

DGNN-based augmentation techniques for the tabular data format may be appropriate for addressing issues related to small-sized consumer survey data because consumer survey data are generally organized into a tabular data format. Several promising algorithms have been developed. Medical GAN (medGAN) is the approach of applying data synthesized by DGNNs to patient data [20]. Table GAN (TGAN) [21], which modifies the deep convolutional GAN [13] for tabular-formed data, is a universal data augmentation approach that is not limited to specified purposes. Conditional table GAN (CTGAN) modifies the TGAN to enable imbalanced data, while the tabular variational autoencoder (TVAE) is an adjusted VAE for the tabular data format [22]. CTGAN and TVAE are not limited to specified purposes, similar to TGAN. Moreover, Xu et al. [17] demonstrated that CTGAN and TVAE outperform other GAN-based synthesizers. They are likely to be effective for this study because they are effective for augmenting tabular-format data in various fields besides consumer survey data [23,24,25]. Generally, most tabular-formed data comprise multiple types of data formats, such as numerical and categorical data. CTGAN and TVAE are modified conventional GAN-based techniques for mixed table-formed data [22,26]. However, they did not consider the dependencies between variables. Several studies have reported that GAN preprocessed by a copula function, known as CopulaGAN, is an appropriate technique for synthesizing data with dependencies among variables [27,28]. Concretely, CopulaGAN is a modification of CTGAN that handles dependencies between variables by applying a copula function to the processes of CTGAN [27]. Because the copula function is introduced to handle the dependencies of price fluctuations between risk assets, such as stocks and bonds, it is applied to a market risk management method called Value-at-Risk in the financial field [29]. Several studies have attempted to generate synthesized asset data using DGNNs [30,31,32]. Harold [32] demonstrated that the synthesized data generated by CopulaGAN reduced volatilities compared with CTGAN and TVAE. Structural equation modeling (SEM) is a well-known statistical analysis method in marketing research [33]. SEM was used to clarify data structures by discerning relationships among variables. Therefore, consumer survey data synthesized using DGNNs might provide information on the dependencies between variables in consumer survey data and enable the handling of mixed table-formed data. This study assesses the abilities of three DGNN-based data synthesizer models—CTGAN, TVAE, and CopulaGAN—on table data with mixed-variable types and on data with dependencies between variables. The characteristics of the three DGNNs are organized in

Table 1. Characteristics of the DGNNs.

	CTGAN	TVAE	CopulaGAN
Mixture table dataset	Yes	Yes	Yes
Dependencies between variables	No	No	Yes

.

Although most questionnaire survey research data are in a tabular data format, no studies have yet reported on their augmentation using DGNNs in marketing research. Moreover, given that the application of DGNNs in marketing contexts is limited to the analysis of scanner panel databases [34] and the production of a product design [35], currently, no study has applied DGNNs to augment questionnaire-based consumer survey data. In addition, augmenting small-sample-sized consumer survey data synthesized by DGNNs might simultaneously solve small-sample-size-related issues (poor marketing decision-making due to limited challenging budgets, a rare population, and the calculation of statistical models) and privacy issues. In particular, the results of this study might contribute to planning marketing strategies for small-sized organizations with limited marketing research budgets. Even with a limited budget, augmenting small-sample-sized consumer survey data might enable us to make marketing plans quantitatively and objectively, similar to blue-chip companies such as P&G and IBM. In this study, we attempted to address these issues by employing a data augmentation method using synthesized data generated by DGNNs. Thus, this study can contribute to enhancing and increasing the use of data augmentation techniques for synthesizing data with DGNNs in several marketing research fields.

2. Methods

2.1. DGNNs

Noncopula (CTGAN and TVAE) and copula DGNN-based data synthesizers can resolve the problems associated with small-sized questionnaire survey data. The DGNNs assessed in this study are described below.

2.1.1. CTGAN

Like the other GAN algorithms, the CTGAN architecture primarily comprises two modules: generator and discriminator (Figure 1A). Here, $g$ denotes the generator responsible for producing fake data, whereas C(∙) represents the discriminator tasked with distinguishing between fake and real inputted data. The loss function is denoted by $\mathcal{L}$(∙), and z signifies the noise generated from a standard normal distribution (N(0,1)). Before being fed into the critic module, the seed data undergo normalization using mode-specific normalization methods.

Both ${\alpha }_{i,j}$ and ${\beta }_{i,j}$ correspond to values associated with continuous columns in the seed data, whereas ${\mathit{d}}_{i,j}$ represents a one-hot vector denoting a discrete value (${\mathit{d}}_{i,j}$) of the i-th discrete column in the j-th row of the seed data. These values, namely, ${\alpha }_{i,j}$, ${\beta }_{i,j}$, and ${\mathit{d}}_{i,j}$, are generated using the generator, which is constructed using a neural network architecture. The neural network architecture is detailed below (Equations (1)–(5)).

$$h_1=ReLU\left(BN\right({FC}_{u \to v }(z \oplus cond)\left)\right),u=\left|cond\right|+\left|z\right|, v=256$$

(1)

$$h_2=ReLU\left(BN\right({FC}_{u\to v }(z \oplus cond\oplus h_1)\left)\right),u=\left|cond\right|+\left|z\right|+256, v=256$$

(2)

$$\widehat{{\alpha }_i}=tanh\left({FC}_{u\to v }\left(h_2\right)\right),1\le i \le N_c,u=\left|cond\right|+\left|z\right|+512, v=1$$

(3)

$$\widehat{{\beta }_i}={gambel}_{\tau }\left({FC}_{u\to v }\right(h_2\left)\right),1 \le i \le N_c,u=\left|cond\right|+\left|z\right|+512, v=m_i$$

(4)

$$\widehat{{\mathit{d}}_i}={gambel}_{\tau }\left({FC}_{u\to v}\left(h_2\right)\right),1\le i \le N_d,u=\left|cond\right|+\left|z\right|+512, v= \left|D_i\right|$$

(5)

The term $h_i$ refers to a hidden layer within the neural network concerning the function types of layers within the neural network; BN and FC denote batch normalization and fully connected layers, respectively. The variables u and v indicate the number of neurons in the input and output layers, respectively. “cond” means the conditional vectors. $\widehat{{\alpha }_i}, \widehat{{\beta }_i}, \mathrm{a}\mathrm{n}\mathrm{d} \widehat{{\mathit{d}}_i}$ are activation functions in the l-th fully connected layers and ReLU, tanh, and gumbel denote specific activation functions (Leaky Rectified Linear Unit, hyperbolic tangent, and Gumbel, respectively). The parameter τ is associated with the Gumbel function. $N_c$ and $N_d$ represent the numbers of continuous-data and discrete-data columns, respectively. $D_i$ is the ith discrete column. $m_i$ corresponds with $D_i$ and the ith musk vector consisting of 0 and 1. CTGAN uses the Wasserstein loss function [36], unlike the conventional GANs. This choice mitigates the mode-dropping phenomenon. The Adaptive moment estimation (Adam) optimizer is used to optimize the Wasserstein loss function.

$$\mathcal{L}\left(·\right)=E_{x~X, z~N(o,\mathit{I})}\left[C\left(x\right)\right.-C\left(g\left(z\right)\right)+\left.regulazation term\right]$$

(6)

x is data in the seed data X. z represents random vectors taken from $\mathrm{N}(o,\mathit{I})$. $\mathcal{L}\left(·\right)$ is the loss function. $C(·)$ is the critic. $g$(∙) is the generator.

2.1.2. TVAE

The TVAE shares the same architecture as the VAE (Figure 1B) [12], and the neural network model is detailed below (Equations (7)–(19)). Here, $E(·)$ represents the probabilistic encoder, whereas $D(·)$ denotes the probabilistic decoder. The encoder and decoder are implemented using neural network models. These two neural networks are described below.

$${\mathit{r}}_j=cat ({\alpha }_{1,j}{, \beta }_{1,j}, \cdots , {\alpha }_{N_c,j}{, \beta }_{N_c,j}, {\mathit{d}}_{1,j }\cdots , {\mathit{d}}_{N_d,j})$$

(7)

$$h_1=ReLU\left(BN\right({FC}_{u\to v }\left({\mathit{r}}_j\right)),u={\mathit{r}}_j, v=128$$

(8)

$$h_2=ReLU\left(BN\right({FC}_{u\to v }\left(h_1\right)),u=128, v=128$$

(9)

$$\mu ={FC}_{u\to v }\left(h_2\right)),u=128, v=128$$

(10)

$$\sigma =exp({{\displaystyle \frac{1}{2}}FC}_{u\to v }(h_2)),u=128, v=128$$

(11)

$$q_{\varphi }\left(z_j|{\mathit{r}}_j\right) ~ N(\mu , \sigma \mathit{I})$$

(12)

The encoder network is similar to that of a variational autoencoder (VAE). Symbols such as ${\alpha }_{i,j}$, ${\beta }_{i,j}$, ${\mathit{d}}_{ij},h_i$, $N_c, N_d,$BN, FC, u, v, and ReLU correspond to components within the network architecture, particularly in the context of CTGAN. The term “cat” represents categorical data. The symbols μ and σ denote vectors representing location and scale parameters within the hidden layers, respectively, with “exp” representing the exponential function. The variable ${\mathit{r}}_j$ refers to rows within tabular data T, where ${\mathit{r}}_j$’s involve concatenating continuous and discrete data within the same tabular dataset T. z represents a hidden representation, i.e., hidden vectors sampled from a Gaussian distribution ($\mu +\sigma \odot \epsilon , \epsilon =\mathrm{N}(0, \mathit{I})$). A detailed description is given in Figure 1. The expression $q_{\varphi }\left(z_j|{\mathit{r}}_j\right)$ denotes the probabilistic encoder model (Equation (12)), assuming a multivariate Gaussian distribution N ($\mu , \sigma \mathit{I}$).

Concerning the decoder network, ${\overline{\alpha }}_{i,j}$ denotes values computed by the activation function, whereas ${\widehat{\alpha }}_{i,j}$, ${\widehat{\beta }}_{i,j}$, and ${\widehat{\mathit{d}}}_{i,j}$ denote random variables. ${\delta }_i$ represents network parameters, with “softmax” indicating the softmax function. The expression $p_{\theta }\left({\mathit{r}}_j|z_j\right)$ denotes the probabilistic decoder model (Equation (19)), assuming a probability distribution denoted by $P\left(·\right)$.

$$h_1=ReLU\left(BN\right({FC}_{u\to v }\left(z_j\right)), u=128, v=128$$

(13)

$$h_2=ReLU\left(BN\right({FC}_{u\to v }\left(h_1\right)),u=128, v=128$$

(14)

$${\overline{\alpha }}_{i,j}={tanh(FC}_{u\to v }\left(h_2\right)),1\le i \le N_c,u=128, v=1$$

(15)

$${\widehat{\alpha }}_{i,j}~ \mathrm{N}({\overline{\alpha }}_{i,j}, {\delta }_i),1\le i \le N_c$$

(16)

$${\widehat{\beta }}_{i,j}={softmax(FC}_{u\to v }\left(h_2\right)),1\le i \le N_c,u=128, v=m_i$$

(17)

$${\widehat{\mathit{d}}}_{i,j}={softmax(FC}_{u\to v }\left(h_2\right)),1\le i \le N_d,u=128, v=\left|D_i\right|$$

(18)

$$p_{\theta }\left({\mathit{r}}_j|z_j\right)={\prod }_{i=1}^{N_c}P ({\widehat{\alpha }}_{i,j}-{\alpha }_{i,j}) {\prod }_{i=1}^{N_c}P ({\widehat{\beta }}_{i,j}-{\beta }_{i,j}){\prod }_{i=1}^{N_d}P ({\widehat{\mathit{d}}}_{i,j}-{\mathit{d}}_{i,j})$$

(19)

Because $p_{\theta }\left({\mathit{r}}_j|z_j\right)$ is a joint distribution, the formula can be transformed into an additive model, as described below. $CE\left(·\right)$ denotes the conditional expected loss. The weights of activation functions and ${\delta }_i$ are trained using the Adam algorithm.

$${log p}_{\theta }\left({\mathit{r}}_j|z_j\right)=\log {\displaystyle \frac{1}{\sqrt{2\pi {\delta }_i}}}exp{\displaystyle \frac{({\alpha }_{i,j}-{\overline{\alpha }}_{i,j})}{{2\delta }_i^2}}+{\sum }_{i=1}^{N_c}CE\left({\widehat{\beta }}_{i,j}, {\beta }_{i,j}\right)+{\sum }_{i=1}^{N_d}CE\left({\widehat{\mathit{d}}}_{i,j} {\mathit{d}}_{i,j}\right)+const.$$

(20)

2.1.3. CopulaGAN

In CopulaGAN, the dependencies between columns in the input data are preprocessed using the Gaussian copula function as the cumulative distribution in the prior phase of data generation. The cumulative distribution functions of columns 0–n are expressed as $F_0$, $\cdots$ $F_n$. Elements of each row are denoted as X = ($x_0$, $\cdots$ $x_n$) in vector form. $F_i\left(x_i\right)$ is the marginal distribution function. The Gaussian copula function Y is defined below (Equation (21)). ${\mathsf{\Phi }}_n(;\mathsf{\Sigma })$ is the joint distribution function of the n-variate Gaussian distribution with parameters $\mu , \sigma ,$and $\mathsf{\Sigma }($coefficient matrix). ${\mathsf{\Phi }}_1^{-1} (·)$ is the inverse of the univariate cumulative distribution function of the Gaussian distribution. After all rows are processed, each parameter in Equation (22) is estimated and the covariance matrix $\mathsf{\Sigma }$ is calculated. Information on $\mu , \sigma ,$and $\mathsf{\Sigma }$ is used to generate synthesized data. The remaining processes are the same as those in CTGAN.

$${\mathrm{Y}=\mathsf{\Phi }}_n({\mathsf{\Phi }}_1^{-1}\left(F_0\left(x_0\right)\right),{\mathsf{\Phi }}_1^{-1}\left(F_1\left(x_1\right)\right),\cdots ,{\mathsf{\Phi }}_1^{-1}\left(F_n\left(x_n\right)\right);\mathsf{\Sigma }) {=\mathsf{\Phi }}_n({\displaystyle \frac{x_0-{\mu }_0}{{\sigma }_0}},\cdots ,{\displaystyle \frac{x_n-{\mu }_n}{{\sigma }_n}};\mathsf{\Sigma })$$

(21)

$$F_i\left(x_i\right)={\mathsf{\Phi }}_1 ({\displaystyle \frac{x_i-{\mu }_i}{{\sigma }_i}}), 0\le i\le n$$

(22)

2.2. Questionnaire Survey Research

We conducted questionnaire survey research to validate the augmented data based on the synthesized data generated by the DGNNs. A survey evaluating product images under the category of motorbikes made by manufacturers from China and Japan was performed online, with Japan residents as the participants. We designed 26 questionnaires related to attributes for product evaluation in each country. The respondents were required to rate each questionnaire on the Likert scale from 1 to 11. Table S1 (“S” means Supplementary Materials. Refer to the Supplementary Materials section) describes the detailed questionnaires. The survey in this study was executed through the Internet based on a secured survey panel database provided by the marketing research agency. We cannot access the detailed data of respondents because the data are strictly managed by marketing research agency officers. When conducting this survey, the agency confirmed each respondent’s approval of using their data for this study.

2.3. Validation Procedures

2.3.1. Dataset

Eleven datasets were classified into two large groups: “Nonaugmented” and “Augmented” datasets (Figure 2). The “Nonaugmented” dataset comprises both real and synthesized datasets generated by DGNNs (CTGAN, TVAE, and CopulaGAN). The real dataset comprises the original and seed datasets. The original dataset is a large-sample-sized dataset used as a reference (Table S10). It is an innately desirable dataset for making marketing decisions. The seed data comprise a small-sample-sized dataset that is the seed for synthesizing data using DGNNs (Table S11). Seed data are illustrative and express a dataset obtained under severe and limited marketing research budgets. The other datasets are synthesized datasets generated by each DGNN. The “Augmented” dataset comprises six datasets that combine the real and synthesized datasets. Concretely, the original or seed dataset and any of the synthesized datasets are combined. Concerning the effectiveness of data augmentation in small-sample-sized survey data, the combined seed and synthesized datasets are used.

2.3.2. Validation Approach

We validated the effectiveness of the synthesized data following two steps (Figure 3). First, we assessed the quality of the synthesized data. Second, we evaluated the data augmentation practicability using the assessed synthesized data from the previous step. Outcome prediction and feature evaluation play crucial roles in marketing analysis, similar to the application of machine learning techniques aside from marketing research. Outcome prediction is applied to predict preferred brands, while feature evaluation is used to identify influential variables toward outcomes. The latter analysis is particularly referred to as the key driver analysis in the marketing analysis, playing a crucial role in marketing decision-making [37]. Thus, assessing the predictive performance and the feature importance is necessary in validating the usefulness of data augmentation with the synthesized data.

2.3.3. Assessing the Synthesized Data Accuracy

We assessed the synthesized data accuracy by adopting the following three metrics: column shape, column pair trend, and overall quality score [27].

The column shape score covers the 0.0 to 1.0 metrics. Column shape 1.0 shows that both data have completely equal distributions. This metric represents the similarity in the column distributions in each datum. The column shape formula is presented as follows:

$$Column shape score=1-(KS statistic)$$

(23)

The KS statistic represents the values calculated by the Kolmogorov–Smirnov test.

The column pair trend score concerns the metrics using correlation coefficients. The best value for the column pair trend is 1.0, whereas the worst value is 0.0. This metric represents the structured similarities between the two data. The column pair trend formula is as follows:

$$Column pair trend score=1-{\displaystyle \frac{\left|S_{A, B}-R_{A, B}\right|}{2}}$$

(24)

S represents the synthesized data; R represents the original data; and A and B are the columns in each dataset. $S_{A,B}$ represents the correlation values of the synthesized data, and $R_{A,B}$ represents the correlation values of the original data.

The overall quality score is the value calculated by averaging both the column shape score and the column pair trend score.

$$Overall quality score={\displaystyle \frac{Column shape score+Column pair trend score}{2}}$$

(25)

2.3.4. Evaluating the Practicability of Data Augmentation

We evaluated the practicability of data augmentation by defining the questionnaires from Q1 to Q26 as the features and Japan or China as the outcome. We addressed the tackled issue by using classification task algorithms in machine learning.

Even if the synthesized data can accurately be generated in terms of data quality, their high quality alone would not necessarily be practical when predicting the outcomes and evaluating the contributions to the outcomes based on the synthesized data. As for the latter, the contribution to the outcome is measured using feature importance scores. The feature importance scores calculated based on large-sample-sized data are considered credible. Given that we refer to the feature importance scores calculated on the original data as the benchmark in this work, we evaluated the degree of concordance between the feature importance scores for the original data and those for the other data. We assessed the practicability of data augmentation in terms of both the predictive performance and the concordance of the feature importance scores. Based on several previous studies [22,24,38,39], the efficacy of the 11 prepared datasets was validated by conducting experiments using several major machine learning algorithms (i.e., boosting, random forest, support vector machine <radial-based function>, and support vector machine <linear function>) in JASP 0.18.1 [40]. The performances of these machine learning algorithms with the prevailing evaluation metrics (i.e., accuracy, precision, recall, and F1 score) were evaluated.

Regarding the predictive performance, we calculated the averages of the evaluation metrics such as accuracy, precision, recall, F1 scores, and area under the curve (AUC) in each machine learning algorithm using each datum and its standard deviation. The average values of these evaluation metrics represent the predictive performance when using the data, while the standard deviation values depict the predictive performance stability. The useful data for the predictive analysis were easily assessed by organizing these values into a two-dimensional (2D) map and plotting each datum in it (Figure 4). The horizontal axis represents the predictive performance, which is the average value of the evaluation metrics, whereas the vertical axis denotes the stability concerning the evaluation metrics when conducting a predictive analysis, which comprises the standard deviation values for the evaluation metrics. These high average and low standard deviation values show the data qualities without depending on the abilities of machine learning algorithms under uniform conditions relative to the hyperparameters of the machine learning algorithms. This map shows that the data with the best predictive performance and the highest stability are the most useful data for prediction tasks in marketing analysis, regardless of the type of machine learning algorithm.

In relation to the degree of concordance of the feature importance scores (Figure 5), the original data, which are the benchmark data, corresponded with Data A, while the other data, such as the seed and synthesized data, corresponded with Data B and C. We calculated the correlation coefficients (Pearson’s r and Spearman’s rho) between the original and other data based on the feature importance scores computed in each machine learning algorithm. These correlation coefficients imply the degree of concordance between the original and other data, validating whether or not the other data offer possibilities of alternatives to the original data when predicting outcomes and assessing the influential features for outcomes. While the data with higher correlation coefficient values might be a prospective alternative to the original data, those with low correlation coefficient values might be less desirable in terms of alternatives to the original data.

To judge which dataset types and data are the most appropriate for marketing analysis, we propose 2D maps that organize the predictive performance results and the degree of concordance by the evaluation metrics (Figure 6). Similar to in Figure 4, the predictive performance was depicted on the horizontal axis. The degree of concordance, which can use the values of the averaged correlation coefficients based on Pearson r and Spearman rho, was set as the vertical axis. Thus, the higher the values on the horizontal axis, the better the results obtained using the concerned data in the machine learning algorithm calculation. The higher values on the vertical axis suggested a close pattern of the feature importance scores between the original and other data. In this map, the data with the best predictive performance and the closest feature importance scores between the original and other data were the most useful data for the marketing analysis.

3. Results

3.1. Questionnaire Survey Research

Table S2 presents the survey results. These survey data are referred to as the real data here. Therefore, we show the survey data results in a split data form of original and seed data. The first acquired survey data were transformed for the consecutive analysis using machine learning algorithms to validate the practicability. Accordingly, considering this consecutive analysis, the descriptive summary in Table S2 is provided based on the transformed data. Figure 7 explains the transformed data. After the initial data transformation, the county column used as an outcome variable in the consecutive machine learning analysis was added.

3.2. Validation Results

3.2.1. Assessing the Synthesized Data Accuracy

We generated the synthesized data based on the seed data using three DGNNs by each epoch. Hyperparameters for three DGNNs are described in Table S3. These DGNNs were developed by the SDV package implemented by Python [27]. We employed the default values used in the SDV package as the hyperparameters. Each DGNN was learned from 10 to 30,000 epochs. We assessed the synthesized data quality using the overall quality score. Although the CTGAN score was slightly increased to the 30,000th epoch, the overall quality score of the GAN-based synthesizers (CTGAN and CopulaGAN) was almost saturated around the 5000th epoch. The TVAE score was saturated around the 3000th epoch. We employed the 30,000th epoch as the best score because each performance index was considered saturated between the 3000th or 5000th and the 30,000th epoch. Therefore, for the synthesized data generated by these synthesizers, the model learned at 30,000 epochs was considered to achieve the best overall quality score.

Table 2. Quality of the synthesized data generated by the DGNNs.

Epochs	Wall Times			Overall Quality Scores			Column Shapes			Column Pair Trends
	CTGAN	TVAE	CP-GAN	CTGAN	TVAE	CP-GAN	CTGAN	TVAE	CP-GAN	CTGAN	TVAE	CP-GAN
10	2.72 s	1.81 s	3.35 s	69.56%	67.07%	66.76%	66.03%	67.48%	62.60%	73.09%	71.83%	70.93%
100	9.23 s	3.06 s	9.64 s	65.98%	79.72%	65.01%	59.00%	73.83%	59.02%	72.96%	85.61%	70.99%
500	43.8 s	8.55 s	35.6 s	78.18%	82.22%	70.81%	69.28%	73.59%	65.71%	87.08%	90.85%	75.91%
1000	1 min 32 s	16.6 s	1 min 8 s	79.02%	82.99%	71.88%	67.07%	73.78%	64.51%	90.97%	92.19%	79.24%
3000	4 min 6 s	47.3 s	3 min 34 s	79.45%	83.51%	78.14%	68.81%	74.61%	66.41%	90.08%	92.40%	89.86%
5000	6 min 55 s	1 min 15 s	5 min 58 s	81.03%	83.56%	80.50%	70.80%	75.05%	70.95%	91.26%	92.07%	90.05%
10,000	16 min 33 s	3 min 3 s	12 min 5 s	81.34%	83.75%	81.77%	71.09%	75.73%	72.16%	91.58%	91.78%	91.38%
20,000	31 min 4 s	5 min 53 s	25 min 19 s	81.98%	83.41%	81.34%	72.09%	74.83%	72.02%	91.86%	91.98%	90.66%
30,000	48 min 32 s	9 min 56 s	37 min 9 s	82.45%	83.93%	82.80%	72.68%	75.70%	74.24%	92.23%	92.15%	91.36%

The bold and underlined values express the best performance within the epochs in each DGNN. Abbreviations: CTGAN, Conditional Table GAN; CP-GAN, CopulaGAN; TVAE, Tabular Variational AutoEncoder.

presents the obtained results. We adopted the two synthesized data for the consecutive analysis. The mixed data (Figure 2) were created by these synthesized data, as well. Table S4 shows a descriptive summary of these three synthesized data. Similar to those in previous studies [41,42], the CTGAN took longer to synthesize the data compared to the TVAE.

3.2.2. Evaluating the Practicability of Data Augmentation

In this study, 11 datasets were created to validate the augmented data (Figure 8). Four machine learning algorithms (i.e., boosting, random forest, support vector machine <radial-based function>, and support vector machine <linear function>) were evaluated on the 11 datasets (Figure 8). The data were split into the following three datasets when these algorithms were run: training, validation, and testing. Then, 20% of the total data were assigned as the test data. The remaining data were the training data. Meanwhile, the data of 20% of the training data were assigned as the validation data. The evaluation metrics were calculated based on the results using the test data. Table S5 describes the parameter setting for each machine learning algorithm, while Table S6 shows the results of these machine learning algorithms.

To assess datasets in terms of their predictive performances and stabilities, the average and standard deviation values of the evaluation metrics across the machine learning algorithms were calculated. The evaluation metrics and confusion matrices are shown in

Table 3. Predictive performances across the machine learning algorithms when using each datum.

Dataset		Evaluation Metrics										Averaged Value of Evaluation Metrics	SD of AVEM
Data Type	Data	Accuracy		Precision		Recall		F1 Score		AUC
		Ave	SD	Ave	SD	Ave	SD	Ave	SD	Ave	SD
Nonaugmented	Real (original)	0.783	0.099	0.797	0.094	0.783	0.099	0.779	0.104	0.813	0.123	0.791	0.104
	Real (seed)	0.788	0.220	0.871	0.095	0.788	0.220	0.755	0.277	0.859	0.211	0.812	0.205
	Synthesized (CTGAN)	0.900	0.097	0.917	0.067	0.900	0.097	0.897	0.103	0.916	0.110	0.906	0.095
	Synthesized (TVAE)	0.852	0.076	0.867	0.052	0.853	0.076	0.851	0.081	0.886	0.105	0.862	0.078
	Synthesized (CopulaGAN)	0.896	0.071	0.905	0.054	0.896	0.071	0.891	0.080	0.906	0.116	0.899	0.079
Augmented	Real (original) + Synthesized (CTGAN)	0.848	0.086	0.859	0.074	0.848	0.085	0.845	0.089	0.873	0.109	0.854	0.089
	Real (original) + Synthesized (TVAE)	0.830	0.060	0.847	0.040	0.830	0.060	0.827	0.065	0.862	0.091	0.839	0.063
	Real (original) + Synthesized (CopulaGAN)	0.886	0.059	0.892	0.047	0.886	0.059	0.870	0.085	0.828	0.168	0.872	0.084
	Real (seed) + Synthesized (CTGAN)	0.896	0.103	0.914	0.068	0.896	0.103	0.893	0.110	0.914	0.115	0.902	0.100
	Real (seed) + Synthesized (TVAE)	0.855	0.067	0.866	0.047	0.855	0.067	0.853	0.071	0.889	0.100	0.863	0.070
	Real (seed) + Synthesized (CopulaGAN)	0.898	0.077	0.908	0.058	0.898	0.077	0.893	0.087	0.895	0.113	0.899	0.082

The bold and underlined values express the best performances, the bold values express the second-ranked performances, and the underlined values depict the third-ranked performances in each column in each datum. Abbreviations: AUC, area under the curve; Ave, average; SD, standard deviation; AVEM, averaged value of evaluation metrics.

and Supplementary Table S7. For the synthesized data in the “Nonaugmented” dataset group, the data generated using the GAN-based synthesizers (CTGAN and CopulaGAN) outperformed those generated by TVAE in terms of predictive performance. These findings align with recent studies [26,43,44,45]. Notably, the real dataset exhibited the lowest performance. Augmented datasets are ranked second or third. Augmenting the original dataset improved the evaluation metrics of predictive performance by an average of 8.16%. Augmenting the seed dataset improved the evaluation metrics of predictive performance an average of 9.64%. The evaluation of the predictive analysis indicated that the GAN-based datasets in the “Nonaugmented” and “Augmented” dataset groups exhibited higher performance levels than the other datasets. In particular, concerning the seed dataset in the “Augmented” dataset group, the combination of the seed and synthesized datasets generated by CTGAN and CopulaGAN improved performance across evaluation metrics by an average of 11.41% and 10.93%, respectively, although TVAE enhanced the performance by 6.58%. The augmented and synthesized datasets had almost the same standard deviations as the original datasets and lower standard deviations than the seed datasets. The TVAE-related datasets had relatively lower standard deviations compared to the other datasets.

Regarding the concordance of the feature importance scores, we performed two correlation analyses (i.e., Pearson and Spearman) for the feature importance scores in each datum. Table S8 shows the results of the feature importance scores in each datum by the machine learning algorithms. The feature importance scores of the support vector machine were not calculated in JASP 0.18.0 [40]; therefore, the scores of these models are not described in Table S8.

Table 4. Concordance analysis: correlation coefficients between the original and other data.

Dataset		Machine Learning Algorithm				Averaged Correlation Coefficients
		Boosting		Random Forest
		Correlation Coefficient
		Pearson	Spearman	Pearson	Spearman
Pairs of Data Types	Compared Data	r	rho	r	rho
Benchmark	Real (original)	1.000 ***	1.000 ***	1.000 ***	1.000 ***	1.000
Original real data vs. Nonaugmented data	Real (seed)	0.751 ***	0.723 ***	0.646 ***	0.683 ***	0.701
	Synthesized (CTGAN)	0.735 ***	0.555 **	0.737 ***	0.747 ***	0.694
	Synthesized (TVAE)	0.672 ***	0.684 ***	0.401 *	0.704 ***	0.615
	Synthesized (CopulaGAN)	0.634 ***	0.770 ***	0.820 ***	0.702 ***	0.732
Original real data vs. Augmented data	Real (original) + Synthesized(CTGAN)	0.821 ***	0.881 ***	0.849 ***	0.837 ***	0.847
	Real (original) + Synthesized (TVAE)	0.779 ***	0.859 ***	0.886 ***	0.890 ***	0.854
	Real (original) + Synthesized (CopulaGAN)	0.020	0.669 ***	0.839 ***	0.710 ***	0.560
	Real (seed) + Synthesized(CTGAN)	0.767 ***	0.518 **	0.706 ***	0.717 ***	0.677
	Real (seed) + Synthesized (TVAE)	0.718 ***	0.692 ***	0.46 *	0.745 ***	0.654
	Real (seed) + Synthesized (CopulaGAN)	0.585 **	0.765 ***	0.839 ***	0.710 ***	0.725

*** p < 0.001. ** p < 0.01. * p < 0.05. The bold and underlined values express the best performances, the bold values express the second-ranked performances, and the underlined values express the third-ranked performances in each column in each data pair.

presents the correlation analysis results between the original and other data. Figure S1A,B and Table S9A,B display the detailed results. Averaged correlation coefficients are the average values of each correlation coefficient across machine learning algorithms in each dataset. A value close to 1 indicates that the orders or numerical values of the variable/feature importance in any dataset resemble those in the original dataset. Most of the datasets yielded averaged correlation coefficient values from 0.500 to approximately 0.700, excluding two “Augmented” group datasets (“Real (original) + Synthesized (CTGAN)” and “Real (original) + Synthesized (TVAE)”). The averaged correlation coefficients in the synthesized datasets (“Nonaugmented” group) were almost the same as those in the “Augmented” group, excluding datasets related to the original dataset. The dataset synthesized by TVAE yielded the worst performance in the concordance analysis, whereas that synthesized by CopulaGAN yielded the highest averaged correlation coefficient among the synthesized datasets. In particular, its Spearman’s rho values were higher than those of the other synthesized and seed datasets. For the “Augmented” group, only the CopulaGAN-related dataset yielded a higher value than the averaged correlation coefficient between the original and seed datasets. This suggests that the concordance with the feature importance between the original and small-sample-sized datasets might improve by 3.42% when augmenting a small-sample-sized dataset with synthesized data generated by CopulaGAN (“Real (original) vs. Real (seed)” = 0.701/“Real (original) vs. Augmented [Real (seed) + Syn (CopulaGAN)]” = 0.725).

The actual predictive performance vs. stability map (Figure 9A) was created based on the columns “Averaged value of evaluation metrics” and “SD of AVEM” in Table 3. The predictive performance vs. concordance of the feature importance score map (Figure 9B) was created based on “Averaged value of evaluation metrics” in Table 2 and “Averaged correlation coefficients” in Table 4. Regarding the averaged correlation coefficient values of the original datasets in Table 4 and Figure 9B, we provided them with 1.00 because we defined them as the benchmark value. From the results of both maps, both augmented and synthesized datasets had higher predictive performance than real datasets, regardless of the data scale. The result of the actual predictive performance vs. stability map (Figure 9A) showed that the augmented and synthesized datasets achieved higher predictive performances than the two real datasets (the original and seed datasets), relatively lower fluctuation than the original dataset, and much lower fluctuation than the seed dataset. Two augmented datasets (“Real (seed) + Synthesized (CTGAN)” and “Real (seed) + Synthesized (CopulaGAN)”) and two synthesized datasets (“Synthesized (CTGAN)” and “Synthesized (CopulaGAN)”) were placed in desirable positions because they yielded high predictive performance and universality regardless of machine learning algorithms. However, the augmented and synthesized datasets had the same or relatively low averaged correlation coefficient values as the seed datasets, unlike the predictive performance. Although the augmented datasets related to the original dataset (Real (original) + Synthesized (CTGAN), Real (original) + Synthesized (TVAE)), which yielded averaged correlation coefficients of approximately 0.850, achieved relatively higher concordance with the original dataset, the values are presumed to be the contribution of the original dataset, not the synthesized dataset generated using each DGNN. Among the augmented data based on the seed data, the augmented data composed of both the seed and synthesized data generated by the CopulaGAN achieved the highest predictive performance and relatively higher concordance.

4. Discussion

To the best of our knowledge, this study represents the first attempt to apply a data augmentation approach to consumer survey data through the usage of synthesized data generated using DGNNs. Our findings revealed that the augmented and synthesized data outperformed real data in terms of predictive performance. This suggests that augmenting or synthesizing data with DGNNs may yield a superior performance in marketing analysis classification tasks, irrespective of the data scale and machine learning algorithms used. Consequently, augmenting and synthesizing small-sample-sized survey data could considerably enhance predictive performance. Moreover, even with a large-sample-sized dataset, if marketers aim to achieve better predictive performance than the current levels, augmenting or synthesizing the dataset could prove effective.

Furthermore, the predictive performance of the synthesized data approximated or surpassed that of the augmented data. The synthesized data generated by CTGAN obtained the highest predictive performance, surpassing that of the augmented data (“Real (seed) + Synthesized (CTGAN)” and “Real (original) + Synthesized (CTGAN)”). A similar trend was observed in datasets related to CopulaGAN and partially in those related to TVAE, indicating that synthesized data can effectively emulate real data as a “clone” of the original dataset. In particular, GAN-based synthesizers (CTGAN and CopulaGAN) were superior to the TVAE synthesizer. The distinctiveness of the network architecture between the GAN-based and VAE-based synthesizers might have caused differences in their predictive performance. The mode-specific normalization module is a unique function in GAN-based synthesizers. It might function effectively when the synthesizers train the data. Given that it adjusted complicated data composed of conditional and continuous columns to an appropriate kernel function using a variational Gaussian mixture model (VGM), ideal data might have been generated based on a probabilistic distribution. However, as this is one assumption, we must investigate this phenomenon further in the following research. Data cloning techniques using synthetic data generated using DGNNs have proven successful in computer vision tasks such as image, movie, and audio recognition [46,47,48,49].

The use of synthesized data as cloned real data may help in addressing security-related concerns in consumer survey research. Respondents often hesitate to disclose personal information regarding wealth and health owing to privacy issues [50], which could impact their willingness to participate in marketing activities. Marketers face challenges in obtaining such sensitive information and may incur higher costs compared with conventional consumer surveys. However, informing respondents in advance that “cloned” survey data synthesized by DGNNs will be used for analysis might alleviate concerns regarding data usage and participation. As synthesized data do not contain specified personal information, even in the event of a data breach, respondents’ privacy remains protected.

Data synthesis techniques can be integrated into the management of highly secure marketing research monitoring databases, such as those containing information on individuals with high incomes or specific medical conditions. This strategy is a promising privacy preserving strategy [51] in the marketing fields. Blockchain technology plays a pivotal role in privacy and security preservation. Joint technologies combining blockchain [52,53] and DGNN-synthesized data [54,55] have offered robust security measures, facilitating the access and utilization of customer databases and thereby enhancing the execution of marketing strategies.

In this way, the present study may offer the possibility to propose promising applications because the predictive performances of synthesized data were better than those of the original data. However, this suggests that there may be discrepancies in the predictive results between synthesized and original data. Although data generated by GAN-based synthesizers led to better results concerning performance indices, the TVAE results showed a closeness between the synthesized and original data. This suggests that TVAE might be a better synthesizer because of its closeness to the original data. The drawbacks identified using these results need to be investigated in future research.

The concordance analysis results revealed a discrepancy between the original data and other datasets, despite the improved predictive performances observed in augmented and synthesized data compared with the real data (seed and original data). This implies that achieving proximity to the values and orders of feature importance in the original data through data augmentation and synthesis is challenging. Consequently, there are concerns regarding the divergence in marketing decisions based on feature importance calculated using augmented and synthesized data compared with those based on the original data. Considering that the degree of concordance of feature importance between the original and other datasets may fall at approximately 85% or lower, marketers should exercise caution when making marketing decisions based on driver analysis, considering these observed discrepancies.

While our findings suggest that reproducing the feature importance observed in the original data presents challenges, augmented and synthesized data related to CopulaGAN exhibited correlation coefficients higher than the average, surpassing 0.7, unlike other DGNNs, which yielded coefficients below 0.7. This indicates that CopulaGAN, with the incorporation of copula function during processing, tends to perform better in achieving concordance with feature importance observed in the original data. This observation aligns with previous studies [28,32], suggesting that CopulaGAN might demonstrate superior performance in concordance analysis compared with other DGNNs, particularly owing to its effectiveness in handling correlated variables commonly found in consumer survey data. Therefore, the synthesized data generated using CopulaGAN might effectively augment data using dependencies between variables in table data other than financial data.

In general, the usage of data augmentation and synthesis techniques can mitigate the reliance on low-credibility data and bolster the implementation of more effective marketing strategies by improving predictive performance. However, a cautious approach may be necessary when assessing the alignment of feature importance through these methods. Implementing the findings of this study into marketing research practices could alleviate the financial burden associated with acquiring large sample sizes and soliciting sensitive information in consumer surveys. In addition, the synthesized data hold promise for establishing a highly accessible and secure consumer survey database. These findings may offer valuable insights for small- and medium-sized organizations with limited marketing budgets seeking to conduct accurate marketing analyses. Our study might present useful findings to practitioners and academicians, but we have several limitations and cautions. In this work, we assessed the usefulness of the data augmentation approach in terms of a classification task alone. Although the classification task is a major approach in marketing analysis, many conventional marketing analysis methods are available (e.g., regression task, factor analysis, and the structural equation model) [56]. Our results are insufficient for application to various marketing issues; thus, further validations regarding the applicability of the data augmentation approach in marketing analysis are needed. The data in this study were well balanced. However, in several marketing research cases, imbalanced data have been observed. Because effective approaches to overcoming imbalanced data issues have been developed to improve the accuracy of deep learning algorithms [57], these approaches need to be applied to handle imbalanced survey data in future studies. Moreover, when it is difficult to apply the current approach, it may be necessary to modify the algorithms in this study with reference to useful approaches to improving deep learning algorithms [58]. Some points must also be considered when applying our validation approach. Our validation approach can be directly applied when the nature of the original data is obvious. However, if the nature is unknown, confirming the generated data quality becomes challenging compared with the original data. Therefore, setting an appropriate number of epochs is hard with the overall quality score, column shapes, and column pair trends. In this situation, machine learning algorithms must be used for all generated data to select the appropriate generated data. Further research is required to confirm the usefulness of data augmentation with the synthesized data generated by the DGNNs in various marketing analysis fields.

In conclusion, applying a data augmentation method using DGNNs for consumer survey data with a small sample size could enhance the predictive performance. Moreover, the predictive performances of the data synthesized by DGNNs were superior to those of the actual data. However, our results also showed that the methods in the current study might enable slight improvement but pose challenges in drastically improving the degree of discrepancy with the feature importance score between large- and small-sample-size data despite augmenting or synthesizing small data. Given that the results using the data generated by CopulaGAN yielded better performance than the other DGNN-based synthesizers, our result might provide a starting point for reducing the discrepancies by considering the correlations between variables in an algorithm.