A Hybrid Deep Learning Approach with Generative Adversarial Network for Credit Card Fraud Detection

Abstract

Credit card fraud detection is a critical challenge in the financial industry, with substantial economic implications. Conventional machine learning (ML) techniques often fail to adapt to evolving fraud patterns and underperform with imbalanced datasets. This study proposes a hybrid deep learning framework that integrates Generative Adversarial Networks (GANs) with Recurrent Neural Networks (RNNs) to enhance fraud detection capabilities. The GAN component generates realistic synthetic fraudulent transactions, addressing data imbalance and enhancing the training set. The discriminator, implemented using various DL architectures, including Simple RNN, Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs), is trained to distinguish between real and synthetic transactions and further fine-tuned to classify transactions as fraudulent or legitimate. Experimental results demonstrate significant improvements over traditional methods, with the GAN-GRU model achieving a sensitivity of 0.992 and specificity of 1.000 on the European credit card dataset. This work highlights the potential of GANs combined with deep learning architectures to provide a more effective and adaptable solution for credit card fraud detection.

1. Introduction

Credit card fraud detection is a critical issue in the financial industry, with substantial economic implications [1,2,3]. According to recent reports, financial institutions worldwide lose over USD 32 billion annually due to fraudulent activities [2,4]. Additionally, the COVID-19 pandemic has led to a rise in online transactions, which further increased the risk of fraud [5]. Therefore, effective fraud detection systems are essential to mitigate these losses and protect consumers.

However, credit card fraud detection presents several challenges that traditional methods and even some DL techniques struggle to address effectively. For instance, fraud patterns constantly evolve as perpetrators develop new strategies to bypass detection systems [6,7]. This continuous evolution requires adaptive models that can learn from new and unseen fraudulent activities in real-time, a task which traditional rule-based systems and even some classical ML models find challenging [8,9,10]. Additionally, credit card fraud is typically a rare event, accounting for a tiny fraction of overall transactions [11]. This leads to highly imbalanced datasets where legitimate transactions significantly outnumber fraudulent ones. Such imbalanced data causes difficulties in training models that can effectively detect the minority class (fraudulent transactions) without overfitting to the majority class [1].

Furthermore, fraudulent behavior is often hidden within sequences of transactions over time. Thus, credit card fraud detection requires models capable of capturing temporal dependencies across sequential data. Fraudulent transactions may exhibit temporal relationships, such as multiple small transactions over a short period or patterns based on transaction time. Recurrent Neural Network (RNN)-based architectures like Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) can model these temporal dependencies effectively. However, even these models face limitations when dealing with extreme data imbalance and complex, evolving fraud patterns that require synthetic data to enhance training.

Therefore, to overcome these challenges, the integration of Generative Adversarial Networks (GANs) into the fraud detection framework introduces a promising solution. GANs can generate realistic synthetic fraudulent transactions to augment the training data, addressing the imbalance issue directly [12]. They can enhance the model’s ability to detect rare events without overfitting the majority class by augmenting the dataset with synthetic examples. Furthermore, combining GANs with RNNs such as LSTM and GRU utilizes the strengths of both models, allowing the model to not only generate realistic synthetic data but also capture temporal dependencies in transaction patterns.

GANs consist of a generator and a discriminator that compete in a zero-sum game. The generator creates synthetic data, while the discriminator attempts to distinguish between real and fake data [13,14]. GANs have achieved improved performance in different applications, including image generation, natural language processing, and anomaly detection [15,16,17]. Therefore, this study aims to employ the robustness of the GAN framework and deep learning (DL) architectures to enhance credit card fraud detection. In the proposed credit card fraud detection approach, the GAN’s generator aims to create synthetic fraudulent transactions. The discriminator, using various architectures such as a Simple RNN, an LSTM, and a GRU, will initially distinguish between real and synthetic transactions. Through this adversarial training process, the discriminator will learn to identify realistic patterns of fraudulent activity. After this initial phase, the discriminator is further fine-tuned to directly classify transactions as fraudulent or legitimate. This dual-phase training process has proven to be effective in different applications [18,19,20]. The rationale for using GANs in conjunction with RNNs is that while RNNs capture temporal dependencies, GANs help balance imbalanced datasets and enhance model generalization by generating synthetic fraudulent transactions.

The main contributions of this study are threefold:

• First, the study proposes a hybrid DL framework that integrates GANs with RNNs to enhance credit card fraud detection. The proposed approach addresses the critical challenge of data imbalance by generating realistic synthetic fraudulent transactions, augmenting the training data and enabling more robust fraud detection.
• Second, the study evaluates multiple RNN architectures, including Simple RNN, LSTM, and GRU, as discriminators within the GAN structure to capture the temporal dependencies inherent in transaction sequences.
• Third, a performance evaluation is conducted on two widely used public datasets to demonstrate the robustness of the proposed framework.

The remainder of this paper is structured as follows: Section 2 reviews related works. Section 3 describes the dataset used in the study, the different DL architecture and the training procedures of the proposed hybrid GAN-RNN models. Section 4 presents the experimental results and discusses the implications of our findings, and Section 5 concludes the paper with suggestions for future work.

2. Related Works

The field of credit card fraud detection has seen significant advancements with the advent of machine learning. Traditional ML techniques, such as decision trees, logistic regression, and support vector machines (SVMs), have been widely applied in predicting credit card fraud. However, they are inefficient in capturing complex, non-linear patterns in transaction data [9,21]. Meanwhile, deep learning has revolutionized many fields due to its ability to model complex data distributions and uncover hidden patterns [22,23,24]. For instance, CNNs have been effectively used to capture spatial features in transaction data, while RNNs and their variants, such as LSTM networks and GRUs, are excellent at modeling temporal dependencies [25]. These approaches have demonstrated significant improvements in detection accuracy compared to traditional methods. However, deep learning models often require large amounts of balanced data for training, which is a common challenge in fraud detection since legitimate transactions often outnumber fraudulent transactions.

Recent advancements in deep learning have led to innovative architectures tailored for specific challenges in image and pattern recognition. For instance, Tao [26] and Tao et al. [27] presented the principles of enhanced deformable convolutions and channel-enhanced networks, respectively, and how they could be adapted to improve the feature extraction capabilities of neural networks used in fraud detection. These studies demonstrate how specialized neural network modifications can significantly enhance detection capabilities in complex environments. Additionally, these adaptations could potentially address the unique challenges of transaction data, which is often non-linear and temporally complex.

Furthermore, a GAN, introduced by Goodfellow et al. [28], has shown remarkable capabilities in generating realistic synthetic data. It consists of two neural networks: a generator, which creates synthetic samples, and a discriminator, which attempts to distinguish between real and synthetic samples. The adversarial training process enables the generator to produce highly realistic data, making GANs useful for tasks such as data augmentation and anomaly detection [17,29,30]. GANs address the problem of imbalanced datasets by generating synthetic fraudulent transactions, providing more training examples for fraud detection models. Recent studies have explored the application of GANs in fraud detection. For instance, Fiore et al. [31] utilized GANs to generate synthetic fraudulent transactions, demonstrating improved detection performance. Similarly, Chen et al. [32] proposed the use of InfoGANs for enhancing fraud detection by generating interpretable synthetic data.

Ding et al. [33] proposed a robust credit card fraud detection system using a hybrid model combining GANs and variational autoencoders (VAEs). The authors generated synthetic fraudulent transactions with a GAN to address the issue of imbalanced datasets, while an autoencoder was employed to reconstruct transaction data and detect anomalies. Their model significantly improved detection performance, especially in terms of recall, compared to other baseline models. Similarly, Wu et al. [34] introduced a GAN-based approach for credit card fraud detection, where the GAN generates synthetic fraudulent transactions, and two autoencoders are used to detect the anomalies, enhancing the model’s ability to detect fraudulent patterns over time. Their results demonstrated that the GAN-autoencoder framework outperformed conventional methods.

Additionally, Banu et al. [35] developed a deep neural network architecture specifically designed for credit card fraud detection by integrating a convolutional neural network (CNN) and an LSTM network. The CNN was employed to capture spatial correlations within transaction features, while the LSTM was used to detect temporal patterns over sequences of transactions. This combination allowed the model to detect fraudulent behavior across both short and long time horizons. Their experimental results demonstrated that this hybrid approach achieved higher accuracy and area under the receiver characteristic curve (AUC) compared to baseline models like random forests and SVMs.

These studies highlight the potential of GANs to improve the robustness and accuracy of fraud detection systems by augmenting the training data with realistic synthetic examples. Furthermore, the combination of GANs and RNNs presents a promising approach to fraud detection by utilizing the strengths of both techniques. GANs can generate synthetic fraudulent transactions that mimic real-world patterns, while RNN models can effectively process and analyze complex transaction data. Few studies have explored this hybrid approach in different applications. For example, Gupta et al. [36] used the GAN-RNN architecture for video generation, while Yang et al. [37] utilized it for health data augmentation.

Therefore, in this study, a hybrid GAN-RNN framework is proposed for effective credit card fraud detection. The framework explores multiple RNN architectures (Simple RNN, LSTM, and GRU) as discriminators within the GAN framework. This study aims to identify the most effective combination for enhancing fraud detection performance by systematically evaluating these architectures.

3. Materials and Methods

3.1. Datasets

In this study, we utilized two widely recognized datasets in the field of credit card fraud detection: the European credit card dataset and the Brazilian credit card dataset. Both datasets are known for their imbalanced nature, with fraudulent transactions constituting a small fraction of the total transactions, which poses a critical challenge for effective fraud detection.

3.1.1. European Credit Card Dataset

The European credit card dataset contains transactions made by European cardholders over a two-day period. This dataset comprises 284,807 transactions, of which only 492 are fraudulent, representing approximately 0.17% of the total transactions [25]. The dataset includes 30 features, such as transaction time, amount, and anonymized principal component analysis (PCA) transformed features. Due to privacy concerns, the actual features are masked, and the anonymized features make it difficult to interpret specific fraud patterns. The highly imbalanced nature of this dataset makes it challenging for ML models to accurately identify fraudulent transactions without overfitting to the majority class. This imbalance highlights the need for approaches like GANs to generate synthetic fraudulent samples for more balanced training data. The dataset is described in

Table 1. European credit card dataset features.

S/N	Feature	Description	Data Type
1	Time	Time elapsed between the first transaction	Numeric
		and the current transaction (in seconds)
2	Amount	Transaction amount	Numeric
3	V1-V28	Anonymized features obtained through	Numeric
		PCA transformation
4	Class	Fraud indicator (1 = Fraud, 0 = No Fraud)	Categorical (0 or 1)

.

3.1.2. Brazilian Credit Card Dataset

The Brazilian credit card dataset was obtained from a large Brazilian bank. This dataset comprises 374,823 transactions, with fraudulent transactions constituting 3.74% of the total [38]. Each transaction includes 17 numerical features, providing detailed transactional information that could aid in fraud detection. The features are shown in

Table 2. Brazilian credit card dataset features.

S/N	Feature	Data Type	Description
1	Time since last transaction	Numeric	Time since the last transaction
2	Amount	Numeric	Transaction amount
3	Type of transaction	Categorical	Card-present or card-not-present
4	Brand/Scheme	Categorical	Card brand (e.g., Visa, MasterCard)
5	Card variant	Categorical	Card type (e.g., local, international)
6	Merchant category code	Numeric	Merchant type code
7	Zip code	Categorical	Zip code of transaction
8	Previous transaction zip code	Categorical	Zip code of previous transaction
9	Previous transaction amount	Numeric	Amount of previous transaction
10	Previous transaction fraud score	Numeric	Fraud score from previous transaction
11	Transaction variant	Categorical	Transaction type (international, local)
12	Importance	Categorical	Cardholder status (VIP or common)
13	Number of installments	Numeric	No. of installments for the transaction
14	Merchant transaction limit	Numeric	Merchant transaction limit
15	Difference in fraud score	Numeric	Difference in fraud score between
			current and previous transactions
16	Time from last transaction	Numeric	Time since last transaction in minutes
17	Class	Categorical	Fraud indicator (1 = Fraud, 0 = No fraud)

. Meanwhile, the imbalance ratio, although lower than the European dataset, still poses a challenge for ML models. The dataset is summarized in Table 2.

Both datasets are extensively used in the research community to benchmark and compare the performance of different fraud detection methods. Their imbalanced nature necessitates the use of advanced techniques such as GANs and DL architectures to effectively learn from the minority class and improve classification performance.

Table 3. Summary of the credit card datasets.

Dataset	Normal	Fraudulent	Features	Sample Size	Fraud (%)
European dataset	284,315	492	30	284,807	0.17
Brazilian dataset	360,792	14,031	17	374,823	3.74

compares and summarizes both datasets.

3.2. Deep Learning Architectures

3.2.1. Simple RNN

A Simple RNN is a type of neural network designed for processing sequential data. Simple RNNs maintain a hidden state that captures information from previous time steps, allowing the network to learn temporal dependencies [39]. The hidden state at each time step t is computed as

$$h_t=\sigma (W_h{h}_{t-1}+W_x{x}_t+b_h)$$

(1)

where $h_t$ is the hidden state, $x_t$ is the input at time step t, $W_h$ and $W_x$ are weight matrices, $b_h$ is the bias term, and $\sigma$ is the activation function [40]. Simple RNNs are useful for tasks that involve time-series or sequential data because they can retain information from previous inputs to inform future predictions. However, they are limited by the vanishing gradient problem, which makes it difficult for them to learn long-term dependencies [41,42]. Despite this limitation, Simple RNNs can be effective for shorter sequences or when combined with other architectures that mitigate their weaknesses.

3.2.2. LSTM

The LSTM network, shown in Figure 1, is an advanced type of RNN designed to overcome the vanishing gradient problem, enabling them to learn long-term dependencies [43]. An LSTM cell contains a cell state $C_t$ and three gates: an input gate $i_t$, a forget gate $f_t$, and an output gate $o_t$. The operations within an LSTM cell are defined as follows:

$$f_t=\sigma (W_f[h_{t-1},x_t]+b_f)$$

(2)

$$i_t=\sigma (W_i[h_{t-1},x_t]+b_i)$$

(3)

$$o_t=\sigma (W_o[h_{t-1},x_t]+b_o)$$

(4)

$$C_t=f_t\ast C_{t-1}+i_t\ast tanh(W_C[h_{t-1},x_t]+b_C)$$

(5)

$$h_t=o_t\ast tanh\left(C_t\right)$$

(6)

where W matrices are weights, b terms are biases, and $\sigma$ and tanh are activation functions [43]. LSTMs are well suited for tasks that require learning long-term dependencies, such as natural language processing and time-series forecasting. Their ability to retain and update information over long sequences makes them ideal for complex sequential patterns where important information might be separated by many time steps [44].

3.2.3. GRU

GRUs are a simplified variant of LSTM networks that also aim to capture long-term dependencies without having the vanishing gradient problem [46]. The GRU architecture is shown in Figure 2. It combines the input and forget gates into a single update gate $z_t$ and uses a reset gate $r_t$. The GRU operations are

$$z_t=\sigma (W_z[h_{t-1},x_t]+b_z)$$

(7)

$$r_t=\sigma (W_r[h_{t-1},x_t]+b_r)$$

(8)

$${\tilde{h}}_t=tanh(W_h[r_t\ast h_{t-1},x_t]+b_h)$$

(9)

$$h_t=(1-z_t)\ast h_{t-1}+z_t\ast {\tilde{h}}_t$$

(10)

By simplifying the forget and input gates into a single gate, the number of parameters and computational complexity are also reduced. This makes GRUs computationally more efficient while still maintaining the ability to capture long-term dependencies [47,48]. They are useful in applications where training time and computational resources are limited, and they often perform comparably to LSTMs on a variety of tasks.

3.3. Generative Adversarial Networks

GANs consist of two neural networks: a generator G and a discriminator D, which compete in a zero-sum game [16]. The generator creates synthetic data, while the discriminator distinguishes between real and fake data. The generator G is designed to produce realistic synthetic fraudulent transactions from a noise vector z:

$$G\left(z\right)=\mathrm{synthetic}\mathrm{transaction}$$

(11)

The generator is trained to minimize the loss:

$${\mathcal{L}}_G=-{\mathbb{E}}_{\mathrm{z}\sim p_{\mathrm{z}}\left(\mathrm{z}\right)}[logD\left(G\left(\mathrm{z}\right)\right)]$$

(12)

Different neural networks can be used as generators, including fully connected neural networks (FCNNs), CNNs, and RNNs. Meanwhile, the discriminator D is trained to classify transactions as real or fake, using architectures such as Simple RNN, LSTM, GRU, and CNN:

$${\mathcal{L}}_D=-{\mathbb{E}}_{\mathrm{x}\sim p_{data}\left(\mathrm{x}\right)}[logD\left(\mathrm{x}\right)]-{\mathbb{E}}_{\mathrm{z}\sim p_{\mathrm{z}}\left(\mathrm{z}\right)}[log(1-D\left(G\left(\mathrm{z}\right)\right)]$$

(13)

The training alternates between updating the discriminator and the generator using stochastic gradient descent. This adversarial training continues until the generator produces highly realistic synthetic transactions and the discriminator accurately distinguishes real from synthetic transactions.

3.4. Proposed Hybrid Deep Learning Framework

The proposed hybrid GAN-RNN framework leverages the generative capabilities of GANs and the sequential modeling strengths of RNNs to enhance credit card fraud detection. The hybrid GAN-RNN framework consists of two main components: the generator and the discriminator. In this study, the generator is implemented using an FCNN. This choice is motivated by the need to effectively generate realistic synthetic fraudulent transactions from tabular credit card transaction data. The FCNN generator starts with a random noise vector and passes it through multiple layers of neurons with ReLU activations to produce synthetic transactions.

Secondly, the discriminator is implemented using different architectures, including Simple RNN, LSTM, and GRU, to evaluate their effectiveness in detecting fraudulent activities. Meanwhile, the discriminator initially focuses on differentiating between real and synthetic transactions. This adversarial process forces the generator to create increasingly realistic fraudulent transactions. Once proficient, the discriminator is fine-tuned to classify transactions as fraudulent or legitimate, utilizing the detailed patterns learned during the adversarial training. The detailed algorithm is shown as Algorithm 1.

Algorithm 1 Hybrid GAN-RNN framework training algorithm.

1: Initialize the generator G and discriminator D with random weights.  2: Set the number of training iterations N.  3: for number of training iterations i from 1 to N do  4:    Step 1: Train Discriminator  5:    Sample a batch of real transactions $x_{real}$ from the dataset.  6:    Sample a batch of random noise vectors z.  7:    Generate a batch of synthetic transactions $x_{fake}=G\left(z\right)$.  8:    Compute discriminator loss: $${\mathcal{L}}_D=-{\mathbb{E}}_{\mathbf{x}\sim p_{data}\left(\mathbf{x}\right)}[logD\left(\mathbf{x}\right)]-{\mathbb{E}}_{\mathbf{z}\sim p_{\mathbf{z}}\left(\mathbf{z}\right)}[log(1-D\left(G\left(\mathbf{z}\right)\right))]$$  9:    Update discriminator D by minimizing ${\mathcal{L}}_D$. 10:    Step 2: Train Generator 11:    Sample a batch of random noise vectors z. 12:    Generate a batch of synthetic transactions $x_{fake}=G\left(z\right)$. 13:    Compute generator loss: $${\mathcal{L}}_G=-{\mathbb{E}}_{\mathbf{z}\sim p_{\mathbf{z}}\left(\mathbf{z}\right)}[logD\left(G\left(\mathbf{z}\right)\right)]$$ 14:    Update generator G by minimizing ${\mathcal{L}}_G$. 15: end for 16: Step 3: Fine-tune Discriminator 17: Fine-tune the discriminator D to classify transactions directly as fraudulent or legitimate using labeled data.

3.5. Evaluation Metrics

This study uses a variety of metrics that provide a comprehensive assessment of the model’s performance. The metrics used are sensitivity, specificity, precision, F-measure, receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC). Sensitivity, also known as recall, measures the proportion of actual fraudulent transactions that the model correctly identifies [49]. It is defined as

$$\mathrm{Sensitivity}\left(\mathrm{Sen}\right)={\displaystyle \frac{TP}{TP+FN}}$$

(14)

where $TP$ is the number of true positives and $FN$ is the number of false negatives. A higher sensitivity indicates that the model is effective in detecting fraudulent transactions [50]. Meanwhile, specificity measures the proportion of actual legitimate transactions that the model correctly identifies [51]:

$$\mathrm{Specificity}\left(\mathrm{Spe}\right)={\displaystyle \frac{TN}{TN+FP}}$$

(15)

where $TN$ is the number of true negatives and $FP$ is the number of false positives. The F-measure is the harmonic mean of precision and recall. It provides a single metric that balances the trade-off between precision and recall. Meanwhile, precision is the proportion of predicted fraudulent transactions that are actually fraudulent:

$$\mathrm{F-measure}={\displaystyle \frac{2·\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(16)

where

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

(17)

The ROC curve is a graphical plot that illustrates the diagnostic ability of the model by plotting the true positive rate against the false positive rate at various threshold settings [3,52]. The AUC provides a single scalar value to summarize the model’s performance. An AUC value closer to 1 indicates a better-performing model.

4. Results and Discussion

This section presents the experimental results and discussion. In the experimental setup of the proposed hybrid GAN-RNN framework, the data are split into training, validation, and test sets using a stratified k-fold cross-validation approach, which is suitable for handling imbalanced datasets. This method ensures that each fold has an equal proportion of fraudulent and legitimate transactions, providing a balanced representation across all splits. Specifically, the data are divided into $k=5$, where $k-1$ folds are used for training, and the remaining fold is used for validation. During the training phase, the GAN’s generator and discriminator are trained iteratively using $k-1$ folds in an adversarial manner.

Once the adversarial training is complete, the discriminator is fine-tuned to classify transactions directly as fraudulent or legitimate using the same $k-1$ folds. This process is repeated k times, with each fold being used as the validation set once. The final model evaluation is performed on an independent test set, which is not used during the training and validation phases. This approach aims to mitigate overfitting and ensures that the model generalizes well to unseen data. Meanwhile, the parameters of the different models used in this study are shown in

Table 4. Model parameters.

Model	Parameters
Simple RNN	units = 128, learning rate = 0.001, optimizer = Adam, loss function = binary_crossentropy, epochs = 20, batch size = 64, activation function = tanh
LSTM	units = 128, learning rate = 0.001, optimizer = Adam, loss function = binary_crossentropy, epochs = 20, batch size = 64, activation function = tanh
GRU	units = 128, learning rate = 0.001, optimizer = Adam, loss function = binary_crossentropy, epochs = 20, batch size = 64, activation function = tanh
GAN	Generator: hidden layer units = 256, activation function = relu Discriminator: Simple RNN/LSTM/GRU, units = 128 learning rate = 0.0002, optimizer = Adam, batch size = 128 loss function = binary_crossentropy, epochs = 100 activation function = sigmoid
Random forest	n_estimators = 100, criterion = entropy, max_depth = 3, min_samples_split = 2, min_samples_leaf = 1
Logistic regression	penalty = l2, C = 1.0, solver = lbfgs, max_iter = 100
SVM	kernel = rbf, C = 1.0, gamma = scale, max_iter = 100
Decision tree	criterion = entropy, max_depth = 4, min_samples_split = 2, min_samples_leaf = 1
XGBoost	n_estimators = 100, learning_rate = 0.1, max_depth = 4, subsample = 0.8, colsample_bytree = 0.8

. These parameters were chosen as they are the commonly used default values in several ML and DL studies, known for their stability and effectiveness in similar binary classification tasks, such as fraud detection [33,53].

4.1. Model Performance Using European Credit Card Dataset

The performance of various models using the European credit card dataset is evaluated and summarized in

Table 5. Performance of the models based on the European dataset.

Model	Sensitivity	Specificity	Precision	F-Measure
GAN-LSTM	0.990	0.995	0.979	0.984
GAN-GRU	0.992	1.000	1.000	0.996
GAN-Simple RNN	0.960	0.991	0.962	0.961
LSTM	0.901	0.990	0.912	0.906
GRU	0.897	0.990	0.850	0.873
Simple RNN	0.884	0.989	0.870	0.877

and Figure 3. The models tested include GAN combined with LSTM, GRU, and Simple RNN, as well as standalone LSTM, GRU, and Simple RNN models. The GAN-GRU model achieved the highest performance among all models, with a sensitivity of 0.992, specificity of 1.000, precision of 1.000, and an F-measure of 0.996. This model’s robustness is further shown by its highest AUC of 0.997, as shown in the ROC curves. These results indicate that the GAN-GRU combination is highly effective in distinguishing between fraudulent and legitimate transactions.

The GAN-LSTM model also performed exceptionally well, with a sensitivity of 0.990, specificity of 0.995, precision of 0.979, an F-measure of 0.984, and AUC of 0.992, demonstrating the effectiveness of combining GAN with LSTM for this application. Meanwhile, the GAN-Simple RNN model, although slightly less effective, still achieved impressive results with a sensitivity of 0.960, specificity of 0.991, precision of 0.962, and an F-measure of 0.961, along with an AUC of 0.987. In contrast, the standalone models (LSTM, GRU, and Simple RNN) showed lower performance. The LSTM model achieved a sensitivity of 0.901, specificity of 0.990, precision of 0.912, and an F-measure of 0.906, with an AUC of 0.971. The GRU model followed closely with a sensitivity of 0.897, specificity of 0.990, precision of 0.850, an F-measure of 0.873, and an AUC of 0.975. The Simple RNN model had the lowest performance, with a sensitivity of 0.884, specificity of 0.989, precision of 0.870, an F-measure of 0.877, and an AUC of 0.955.

Furthermore, the best-performing model (i.e., GAN-GRU) is compared with widely used ML models, including random forest, logistic regression, SVM, multi-layer perceptron (MLP), and extreme gradient boosting (XGBoost). The proposed GAN-GRU is benchmarked against these classifiers in

Table 6. Comparison with baseline models using the European dataset.

Model	Sensitivity	Specificity	Precision	F-Measure
Random forest	0.802	0.931	0.864	0.832
Logistic regression	0.742	0.925	0.811	0.775
SVM	0.579	0.940	0.780	0.665
Decision tree	0.587	0.943	0.830	0.688
MLP	0.730	0.915	0.814	0.769
XGBoost	0.824	0.961	0.887	0.854
GAN-GRU	0.992	1.000	1.000	0.996

.

The results in Table 6 demonstrate the superior performance of the proposed GAN-GRU model compared to widely used ML classifiers. The GAN-GRU model achieved the highest sensitivity, specificity, precision, and F-measure, significantly outperforming traditional models. Among the baseline models, XGBoost showed the best performance, with a sensitivity of 0.824, a specificity of 0.961, and an F-measure of 0.854. However, even XGBoost falls short when compared to GAN-GRU, particularly in terms of sensitivity and precision. Other models, such as random forest and MLP, performed reasonably well but struggled with lower sensitivity values, indicating difficulties in accurately detecting fraudulent transactions. Furthermore, SVM and decision tree showed lower performance in terms of sensitivity and F-measure, highlighting their limited ability to handle the imbalanced nature of the dataset. In contrast, the GAN-GRU model addressed the class imbalance effectively through the generation of synthetic fraudulent transactions, leading to its superior performance across all metrics.

4.2. Model Performance Using the Brazilian Dataset

The performance of the various models using the Brazilian dataset is summarized in

Table 7. Performance of the models using the Brazilian dataset.

Model	Sensitivity	Specificity	Precision	F-Measure
GAN-LSTM	0.920	0.965	0.988	0.953
GAN-GRU	0.903	0.929	0.951	0.926
GAN-Simple RNN	0.898	0.910	0.924	0.911
LSTM	0.715	0.894	0.881	0.789
GRU	0.690	0.918	0.817	0.748
Simple RNN	0.579	0.864	0.703	0.625

and illustrated by the ROC curves in Figure 4. The results consistently demonstrate the superior performance of the GAN-integrated models over their standalone counterparts. Among the tested models, GAN-LSTM obtained the best performance, achieving a sensitivity of 0.920, specificity of 0.965, precision of 0.988, and an F-measure of 0.953. Its AUC of 0.923 further confirms its effectiveness in accurately distinguishing fraudulent transactions from legitimate ones. Similarly, the GAN-GRU model also exhibited strong performance with a sensitivity of 0.903, specificity of 0.929, precision of 0.951, and an F-measure of 0.926, supported by an AUC of 0.913. The GAN-Simple RNN, while slightly behind, still achieved impressive results with a sensitivity of 0.898, specificity of 0.910, precision of 0.924, an F-measure of 0.911, and an AUC of 0.901.

In comparison, the standalone models demonstrated lower performance. The LSTM model recorded a sensitivity of 0.715, a specificity of 0.894, a precision of 0.881, an F-measure of 0.789, and an AUC of 0.819. The GRU model followed with a sensitivity of 0.690, specificity of 0.918, precision of 0.817, and an F-measure of 0.748, alongside an AUC of 0.877. The Simple RNN model scored the lowest in performance, with a sensitivity of 0.579, specificity of 0.864, precision of 0.703, an F-measure of 0.625, and an AUC of 0.791.

Furthermore, the performance of the proposed GAN-LSTM approach is benchmarked with baseline models in

Table 8. Comparison with baseline models using the Brazilian dataset.

Model	Sensitivity	Specificity	Precision	F-Measure
Random forest	0.709	0.885	0.890	0.790
Logistic regression	0.650	0.849	0.820	0.725
SVM	0.586	0.793	0.805	0.678
Decision tree	0.570	0.811	0.786	0.661
MLP	0.592	0.0.820	0.733	0.655
XGBoost	0.764	0.899	0.916	0.833
GAN-LSTM	0.920	0.965	0.988	0.953

. The proposed GAN-LSTM model demonstrates a superior performance compared to the baseline models. The GAN-LSTM achieved the highest sensitivity, specificity, precision, and F-measure, outperforming all the baseline models. Meanwhile, among the baseline models, XGBoost achieved the best performance, with a sensitivity of 0.764, specificity of 0.899, and an F-measure of 0.833. While XGBoost outperforms other baselines in terms of precision, it still falls short compared to GAN-LSTM. Meanwhile, random forest also performed well with a sensitivity of 0.709 and an F-measure of 0.790, but it could not match the overall performance of the GAN-LSTM. The substantial gap in sensitivity and F-measure between the GAN-LSTM and the baseline models demonstrates the effectiveness of combining GANs with RNN architectures. This combination allows the model to better handle imbalanced datasets and capture long-term temporal dependencies, both of which are crucial for detecting fraudulent transactions in highly complex credit card transaction datasets.

4.3. Performance Comparison with Other Scholarly Works

While the proposed approach achieved superior performance compared to the standalone DL and baseline models, it is necessary to compare its performance with state-of-the-art methods in the literature. A comparison is thus tabulated in

Table 9. Comparison with other recent studies.

Reference	Method	Sen	Spe	F-Measure
Xinwei et al. [54]	RNN	-	0.716	0.473
Madhurya et al. [55]	Logistic regression	0.968	0.804	-
Varmedja et al. [56]	MLP + SMOTE	0.816	-	0.804
Awoyemi et al. [57]	Naive Bayes + hybrid sampling	0.807	0.974	-
Alarfaj et al. [58]	XGBoost		-	0.845
Mrozek et al. [59]	Random forest + undersampling	1.000	-	0.118
Forough and Momtazi [2]	LSTM	0.740	-	0.786
Almarshad et al. [60]	GAN	-	-	0.960
Khalid et al. [61]	Logistic regression + SMOTE	0.944	-	0.944
Jain et al. [62]	Random forest	0.780	-	0.858
Lin and Jiang [63]	Autoencoder + random forest	0.814	-
Najadat et al. [64]	Logistic regression	0.720	-	0.220
Forough and Momtazi [38]	LSTM + conditional random fields	0.756	-	0.807
Alwan et al. [65]	Naive Bayes	0.836	-	-
Asha and Kumar [66]	Artificial neural network	0.761	-
Dhankhad et al. [67]	Random forest + undersampling	0.950	0.966	0.950
Dhankhad et al. [67]	XGBoost with undersampling	0.950	0.959	0.950
Alfaiz and Fati [68]	CatBoost + resampling	0.961	-	0.869
Alfaiz and Fati [68]	XGBoost + SVM SMOTE	0.930	-	0.865
Dighe et al. [69]	Logistic regression	0.816	0.962	-
Nadim et al. [70]	Classification and regression trees	0.885	0.910	-
This Paper	Proposed GAN-GRU	0.992	1.000	0.996

, and it is based on studies that used the European dataset. It can be seen that the proposed GAN-GRU model outperformed several models presented in recent studies. The superior performance of our GAN-GRU model, as compared to these recent studies, validates the effectiveness of integrating GANs with the DL architectures to address the challenges of credit card fraud detection, particularly in handling class imbalance and improving model robustness.

4.4. Discussion

The proposed hybrid GAN-RNN framework demonstrated significant improvements in fraud detection performance across the two datasets. The results consistently show that integrating GANs with RNNs, particularly GRUs, enhances the model’s ability to distinguish between fraudulent and legitimate transactions. This can be attributed to the GAN’s capacity to generate realistic synthetic fraudulent transactions, which helps in balancing the training dataset and mitigating the class imbalance issues. The combination of GANs with RNN architectures leverages the strengths of both models. GANs excel in generating realistic data, while RNNs are proficient at capturing temporal dependencies in sequential data. This dual approach allows the model to learn complex patterns associated with fraudulent activities, which standalone models or traditional ML techniques might miss.

Furthermore, our findings align with and exceed the results reported in the existing literature. The superior performance of the GAN-GRU model on the European dataset, with its high sensitivity, specificity, and F-measure, highlights its practical applicability in real-world fraud detection scenarios. The GAN-GRU’s effectiveness on the European dataset can be attributed to its ability to capture short-term dependencies and effectively handle the subtle patterns in the transaction data, which are crucial for this dataset. Conversely, the GAN-LSTM model showed the best performance on the Brazilian dataset. LSTM networks are known for their capability to learn long-term dependencies, which might be more significant given the diverse and detailed features in the Brazilian dataset. The longer period over which transactions were recorded in the Brazilian dataset may benefit more from LSTM’s ability to retain information over extended sequences, helping to identify fraudulent patterns that span over longer periods.

5. Conclusions and Future Research Directions

This study proposed a hybrid DL framework that combines GANs with RNN architectures for credit card fraud detection. The experimental results on two datasets demonstrated the superior performance of the proposed approach, which outperformed other models using the European and Brazilian datasets. This highlights the robustness and effectiveness of the proposed approach in handling class imbalance and enhancing model performance. Our findings indicate that the integration of GANs with DL architectures significantly improves the ability to detect fraudulent transactions, providing a more reliable solution for credit card fraud detection. The dual-phase training process, where the GAN generates synthetic data and the RNN learns from it, proves to be highly effective.

Future research should focus on exploring other GAN configurations and incorporating additional DL models to further enhance the detection capabilities. Moreover, implementing and testing the proposed approach in real-world environments will be essential to validate its practical applicability and operational efficiency. Therefore, addressing challenges related to computational efficiency and scalability will be crucial for deploying these models in large-scale, real-time fraud detection systems.