SynthSecureNet: An Improved Deep Learning Architecture with Application to Intelligent Violence Detection

Abstract

We present a new deep learning architecture, named SynthSecureNet, which hybridizes two popular architectures: MobileNetV2 and ResNetV2. The latter have been shown to be promising in violence detection. The aim of our architecture is to harness the combined strengths of the two known methods for improved accuracy. First, we leverage the pre-trained weights of MobileNetV2 and ResNet50V2 to initialize the network. Next, we fine-tune the network by training it on a dataset of labeled surveillance videos, with a focus on optimizing the fusion process between the two architectures. Experimental results demonstrate a significant improvement in accuracy compared with individual models. MobileNetV2 achieves an accuracy of 90%, while ResNet50V2 achieves a 94% accuracy in violence detection tasks. SynthSecureNet achieves an accuracy of 99.22%, surpassing the performance of individual models. The integration of MobileNetV2 and ResNet50V2 in SynthSecureNet offers a comprehensive solution that addresses the limitations of the existing architectures, paving the way for more effective surveillance and crime prevention strategies.

1. Introduction

The use of closed-circuit television (CCTV) systems is of critical importance in modern society for the purpose of ensuring public safety and security [1,2]. Surveillance systems function as a preventive measure [3] against unlawful activities and improper conduct by means of ongoing monitoring in residential areas, public spaces, transportation hubs, and transportation hubs [4,5,6]. In addition to preventing crime, surveillance technologies also help in quickly identifying and capturing people involved in illegal activities [7]. This enhances the capacity of law enforcement and allows for timely reactions to emerging threats. Furthermore, surveillance data are a crucial kind of forensic evidence in legal procedures, assisting in determining the guilt of criminals and guaranteeing justice for crime victims [8,9]. Surveillance is a fundamental aspect of proactive security measures, enhancing society’s resilience against many possible threats and vulnerabilities [10,11].

Violence detection is one subfield of surveillance that is extremely important to society because it provides an early warning system for identifying and reducing physical violence, conflict, and injury [12,13]. Violence detection systems are able to automatically identify violent incidents on CCTV footage and report them for immediate intervention by utilizing advanced deep learning techniques, such as the proposed SynthSecureNet architecture. The proposed architecture can be integrated with violence systems that do not only allow law enforcement and security personnel to respond quickly and improve public safety, but also give citizens a sense of security and assurance. Furthermore, assisting law enforcement in paying attention to tense situations before they go out of control will reduce chances of people/communities getting harmed, and also violence detection technologies are able to stop conflicts from getting worse. The proactive approach to maintaining peace and order in society that is showcased by the integration of violence detection systems into a surveillance infrastructure highlights the innovative influence of technology on public safety initiatives [14,15,16].

As a result, in this paper, a hybrid deep learning architecture is proposed with enhanced performance when compared with other deep learning architectures for violence detection.

“SynthSecureNet” is a unique name for the proposed ensemble architecture. The term “Synth” suggests synthesis or combining, and “SecureNet” implies a focus on security. This name effectively communicates that the proposed model is a synthesized ensemble designed for security-related tasks.

Zahid et al. [17] proposed a bagging framework known as IBaggedFCNet. This framework applies ensembles for classification and anomaly detection for motion pictures or videos. The authors used a complex Inception-v3 image classification network in their technique and did not use segmentation prior to extracting features from the video. This eliminated the chance for segmentation results from cross interference and reduced the memory usage. They demonstrated significant improvements on different benchmark datasets, with the best improvements recorded when using the UCF-Crime dataset. They conducted several experiments on several ensemble fusion techniques, such as static and dynamic fusions [17,18,19].

The SynthSecureNet architecture stands out from other ensemble architectures based on the combination of the ResNet50V2 and MobileNetV2 architectures. Previous studies have primarily dealt with integrating models that follow similar structural frameworks or using temporal–spatial data separately; on the other hand, SynthSecureNet combines the deep feature extraction strength of ResNet50V2 with the processing speed and light weight of MobileNetV2. This integration also helps to strengthen the model’s ability to recognize potential signs of violence and increases the scope of implementing the system in real time with minimal computational power [20,21,22].

Additionally, the ensemble design for SynthSecureNet is carefully tuned and optimized so that it achieves the best performance in terms of both accuracy and speed. ResNet50V2 is a widely used solution that is ideal for capturing the details and complex patterns in the input data, while MobileNetV2 is designed for specific tasks for edge computing applications, and it has a very effective and simple architecture [23,24]. This combination allows SynthSecureNet to work quicker and achieve better results than other bagging methods that require speed at the expense of accuracy or vice versa. The architecture also implements the strategy for feature fusion and multi-level classification better than the original model so that it can recognize violent actions more intricately and effectively distinguish different actions of violence.

SynthSecureNet has several new qualities that are absent in other violence detection systems. First, it presents a dual-architectural framework that ensures improved flexibility to different surveillance setups, whether they refer to high-resolution urban footage or low-quality video streams from remote locations [21,22]. The fact that the architecture can work effectively on edge devices like smart cameras or mobile units is a step forward because it eliminates the time delay in transmitting data to centralized servers for processing [13].

This hybrid alert system ensures that critical alerts are delivered in real time even in environments where connectivity is limited. The proposed model was trained using a large dataset. Overall, these advancements facilitate the application of SynthSecureNet as a complete predictive system that can comprehensively provide real-time violence detection on CCTV systems for addressing the missing pieces in prior ensemble architectures.

1.1. State-of-the-Art Overview

Recent advancements in deep learning have significantly improved the performance of violence detection systems. Techniques such as ensemble learning, temporal–spatial data modeling, and hybrid architectures have demonstrated their effectiveness. For example, Zahid et al. [17] introduced IBaggedFCNet, a bagging framework leveraging Inception-v3 for anomaly detection. However, this approach lacks segmentation capabilities, which limits its application to complex real-world scenarios. Other approaches, such as Conv3D-based networks, employ temporal feature extraction but are computationally expensive [7]. Despite these advancements, challenges remain in achieving high accuracy while maintaining real-time performance.

1.2. Proposed Contribution

This study presents SynthSecureNet, a novel hybrid architecture combining the strengths of popular MobileNetV2 and ResNet50V2. SynthSecureNet improves violence detection accuracy through feature fusion while maintaining computational efficiency, making it suitable for real-time deployment in edge devices. The architecture achieves a remarkable 99.22% accuracy, surpassing the original methods, and offers a robust solution for public safety initiatives.

2. Methodology

2.1. The Architecture

The SynthSecureNet architecture was built by integrating two well-known deep learning architectures, namely, MobileNetV2 [25] and ResNet50V2 [26]. Our aim was to work on both architectures and fuse them to obtain improved performance for violence detection in videos. MobileNetV2 is a convolutional neural network architecture that is based on the previous multiple versions of MobileNets, which aims to design more efficient deep networks used for mobile and edge devices [27]. It uses the concept of depthwise separable convolutions and inverted residual blocks, which helps in minimizing the number of parameters and computational cost with high performance.

ResNet50V2 (short for residual networks) introduces residual connections to deep Convolutional Networks, which helps in solving a vanishing gradient issue [28]. It is a series of residual blocks. That is where the ‘residual’ in ‘ResNet’ comes from because each block makes it very easy to learn residual functions with respect to the layer inputs and help you create much deeper networks [29].

Although state-of-the-art methods such as SlowFast [30], ResNet 3D [31,32], and Video Swin Transformer [33] offer advanced temporal–spatial modeling for video-based tasks, their inclusion was not feasible in this study due to computational limitations. These methods require extensive hardware resources, which were beyond the scope of the current experimental setup. Future research will benchmark SynthSecureNet against these methods to further validate its performance and adaptability.

2.2. Combining ResNet and MobileNet

As stated in the above paragraph, the SynthSecureNet architecture is basically a combination of two architectures, namely, ResNetV2 and MobileNetV2 architectures. When initializing the model, MobileNetV2 and ResNet50V2 are initialized through pre-trained weights on ImageNet data. The pre-trained weights are a great starting point for feature extraction as they have learned useful representations from a large-scale image classification problem [34,35].

Both MobileNetV2 and ResNet50V2 models are used as feature extractors, which involves removing their classification layers and utilizing the outputs from their last convolutional layers as feature maps. For a better explanation, let $f_m\left(x\right)$ and $f_r\left(x\right)$ depict the feature maps extracted by MobileNetV2 and ResNet50V2, respectively, for an input x.

Along the channel dimension, features extracted from MobileNetV2 and ResNet50V2 were concatenated. This concatenation ensures that the model learns from the non-duplicated information in both architectures. The aforementioned concatenated feature map $f_c\left(x\right)$ can be mathematically shown as (1). The number of channels on the resulting feature map $f_c\left(x\right)$ is greater than single feature maps [34,36].

$$f_c\left(x\right)=contat\left(f_m\left(x\right),f_r\left(x\right)\right)$$

(1)

Then, the concatenated feature map $f_c\left(x\right)$ is processed by extra layers to extract higher-level representations and perform classification. Additional layers include a flattened layer where the spatial feature map is converted into a one-dimensional vector. Additional layers also include dense layers, which are fully connected and learn non-linear combinations of features. Further, they also include dropout layers, which are a technique that performs regulation to prevent overfitting, where units are dropped out during the training duration [37].

This can also be denoted as the output of those extra layers (2), as follows:

$$y=g\left(f_c\left(x\right)\right)$$

(2)

The function of $g\left(\right)$ corresponds to the mapping learned by additional layers, which takes the input as a concatenated feature map $f_c\left(x\right)$ and the output as y.

On the output layer, a sigmoid activation function is used in the final dense layer that produces binary output to classify whether the video contains violence or not [38,39]. Here, the output is squashed between 0 and 1; hence, it can be considered as a probability value. The output of the SynthSecureNet model can be defined as follows (3):

$$p\left(y\mid x\right)=\sigma \left(w·g\left(f_c\left(x\right)\right)+b\right)$$

(3)

where

w is a weight vector;

b is a bias term;

$w·g\left(fc\right(x\left)\right)+b$ is a linear combination of the transformed features.

We can consider the following sigma equation:

$$\sigma \left(z\right)={\displaystyle \frac{1}{1+e^{-z}}}$$

(4)

Thus,

$$p\left(y\mid x\right)={\displaystyle \frac{1}{1+e^{-(w·g\left(f_c\left(x\right)\right)+b)}}}$$

(5)

This equation models the probability $p\left(y\right|x)$ as a sigmoid function of the linear transformation of the features obtained from the concatenated outputs of MobileNetV2 and ResNetV2 [38]. The function corresponds to the mapping learned by additional layers, which takes the input of concatenated features.

The SynthSecureNet model was trained using the binary cross-entropy loss function, which measured the dissimilarity between the predicted probabilities and the true labels. The Adam optimizer was used to update the model’s weights based on the gradients computed during backpropagation [40,41].

2.3. SynthSecureNet Architecture Network Development

The input to the SynthSecureNet model is a video, which undergoes further preprocessing and is converted into a sequence of frames. The input video is denoted by V, whereas N represents the number of frames in it; refer to the following Equation (5):

$$V=\left\{f_1,f_2,\dots ,f_N\right\}$$

(6)

where $f_i$ is an image (each frame).

In a preprocessing phase, before exposure to the model, the frame (or stack of frames) goes through data augmentation processing that increases training sample diversity and improves generalization. Some of these techniques that were used in augmentation include the following [18]:

• Random flipping;
• Random zooming;
• Random brightness adjustment;
• Random rotation.

For better explanation purposes, we denote an augmented frame as $f_i^{aug}$.

In the feature extraction phase of the proposed system, the augmented frames were then passed through MobileNetV2 and ResNet50V2 to extract the features. These models were initialized with pre-trained weights of what is commonly known as ImageNet dataset [29].

MobileNetV2 is made up of a series of depthwise separable convolutions and inverted residual blocks.

The output of the MobileNetV2 model for the frame $f_i^{aug}$ is denoted as follows:

$$f_m\left(f_i^{aug}\right)=MobileNetV2\left(f_i^{aug}\right)$$

(7)

where the feature map $f_m\left(f_i^{aug}\right)$ has the shape of $(h_m,w_m,c_m)$, where $h_m$, $w_m$, and $c_m$ are the height, width, and number of channels, respectively.

Given that the ResNet50V2 model is made up of a sequence of residual blocks, each block learns a residual function.

The output of this ResNet50V2 model for the frame $f_i^{aug}$ can be written as follows:

$$f_r\left(f_i^{aug}\right)=ResNet50V2\left(f_i^{aug}\right)$$

(8)

where the feature map $f_r\left(f_i^{aug}\right)$ has the shape of $(h_r,w_r,c_r)$, where $h_r,w_r,$ and $c_r$ are the height, width, and number of channels, respectively.

The final feature map is acquired by concatenating feature maps from MobileNetV2 and ResNet50V2 along the channel dimension. The concatenation operation can be represented as follows:

$$f_c\left(f_i^{aug}\right)=concat\left(f_m\left(f_i^{aug}\right),f_r\left(f_i^{aug}\right)\right)$$

(9)

The resulting feature map $f_c\left(f_i^{aug}\right)$ has the shape of $(h_c,w_c,c_c)$, where $h_c=min(h_m,h_r),$$w_c=min(w_m,w_r)$, and $c_c=c_m+c_r$.

The concatenated feature map $f_c\left(f_i^{aug}\right)$ is passed through additional layers to learn higher-level representations and perform classification [34]. Then the flattened layer converts the spatial feature map into a 1D vector. The output of the flattened layer can be represented as follows:

$$v_i=flatten\left(f_c\left(f_i^{aug}\right)\right)$$

(10)

where the flattened vector $v_i$ has the shape of $(c_c\times h_c\times w_c)$.

In dense layers for the proposed model, the flattened vector $v_i$ is passed through a series of dense (fully connected) layers.

Each dense layer applies a linear transformation followed by a non-linear activation function like ReLU.

The output of the j-th dense layer can be represented as follows:

$$d_j\left(v_i\right)=ReLU\left(W_j•v_i+b_j\right)$$

(11)

Here, $W_j$ and $b_j$ are the learnable weights and biases of the j-th dense layer, respectively.

Dropout is implemented for regularization where some of the neurons/synapses are omitted randomly during the training process in a given iteration to overcome overfitting. The output of the dropout layer can be represented as follows:

$$d_j^{drop}\left(v_i\right)=dropout\left(d_j\left(v_i\right),rate\right)$$

(12)

where rate is the dropout rate, specifying the fraction of units to be dropped.

In an output layer, after the final dense layer, the output goes through a sigmoid activation function in order to make a binary prediction, as follows:

$$p\left(y_i\right|f_i^{aug})=sigmoid\left(W_{out}•d_{final}\left(v_i\right)+b_{out}\right)=\sigma \left(W_{out}•d_{final}\left(v_i\right)+b_{out}\right)$$

(13)

where $W_{out}$ and $b_{out}$ are the learnable weights and bias of the output layer, respectively. The sigmoid function is defined as as follows:

$$\sigma \left(z\right)={\displaystyle \frac{1}{1+e^{-z}}}$$

(14)

During the training process, we trained the proposed SynthSecureNet model with the binary cross-entropy loss function, which is computed according to the predicted probabilities and the true labels. The loss for a single frame $f_i^{aug}$ can be computed as follows:

$$L(y_i,p\left(y_i\right|f_i^{aug}))=-[y_i•log\left(p\left(y_i\right|f_i^{aug}\right))+(1-y_i)•log(1-p\left(y_i\right|f_i^{aug})\left)\right]$$

(15)

where the $y_i•log\left(p\left(y_i\right|f_i^{aug}\right))$ term is large (more negative) when the true label $y_i$ is 1 and the predicted probability $p\left(y_i\right|f_i^{aug})$ is close to 0.

Additionally, the $(1-y_i)•log(1-p\left(y_i\right|f_i^{aug}))$ term is large (more negative) when the true label $y_i$ is 0 and the predicted probability $1-p\left(y_i\right|f_i^{aug})$ is close to 0. The overall loss $L(y_i,p\left(y_i\right|f_i^{aug}))$ is the sum of these two terms, ensuring that the model is penalized for making incorrect predictions, and the penalty increases as the prediction confidence in the wrong direction increases for the proposed model.

The overall loss for the entire video is the average of the frame-level losses as shown in the above equation.

$$L={\displaystyle \frac{1}{N}}\sum _{i=1}^{N}L(y_i,p\left(y_i\right|f_i^{aug}))$$

(16)

The weights for the proposed model were updated using the Adam optimizer, which computed adaptive learning rates for each parameter based on the first and second moments of the gradients [40,42].

In the inference stage, the preprocessed frames of a video are passed through the trained SynthSecureNet model for obtaining frame-level predictions. The final prediction for each video is initialized on top of the frame-level predictions by majority averaging.

2.4. Ensemble Model (Benefits)

Ensemble models are preferred in many machine learning tasks compared with individual models. Due to their performance that surpasses the performance of an individual model, the benefits of using ensemble models are briefly discussed in the paragraphs below.

2.4.1. Better Predictive Performance

Ensemble models have the capacity to outperform individual models in terms of prediction. Ensembling generally decreases inaccuracy compared with utilizing a single model for making predictions. Ensemble models achieve this by gaining and leveraging the optimal characteristics of each individual model, reducing the flaws of each model [43,44,45].

2.4.2. Less Overfitting

Overfitting arises when a model acquires the ability to perfectly match the random fluctuations included in the training data, leading to inadequate performance when applied to new, unseen data. Ensemble methods can mitigate overfitting by combining the predictions of numerous models in a way that achieves a more accurate final prediction. Each individual model may exhibit its own method of overfitting the training data. However, when the predictions of the ensemble are combined, the ensemble demonstrates superior generalization performance. Ensembling, which combines multiple models together, effectively mitigates any overfitting tendencies exhibited by individual models [46].

To address overfitting, dropout layers were incorporated into the SynthSecureNet architecture. This technique randomly drops units during training, helping to generalize the model to unseen data. However, other advanced methods for preventing overfitting, such as early stopping and advanced regularization techniques (e.g., L2 regularization, weight decay), were not utilized in this study. Additionally, ensemble diversity metrics, which measure the uniqueness and complementarity of the models combined in an ensemble, could further enhance the robustness of SynthSecureNet. These methods will be explored in future work to improve the model’s performance and generalization.

2.4.3. Less Increased Robustness

Ensemble models have demonstrated greater resilience to noise, outliers, and fluctuations in the data as compared with individual models. While individual models may be susceptible to errors or highly influenced by certain data points, the ensemble can nevertheless yield meaningful predictions when the majority (or all) of the models demonstrate strong performance. The predictions generated by the ensemble are less likely to be as vulnerable as those generated through an individual model [47].

2.4.4. Dealing with Complex Relationships

Ensemble models have the ability to acquire intricate linkages and patterns within the data. The aspects or collections of the data that one individual model accurately represents may differ from those of the others. By combining these models, the ensemble can impart a superior and more intricate depiction of the authentic patterns [48]. Ensemble approaches address non-linear linkages, interactions, and dependencies that individual models may find challenging [49,50,51].

2.4.5. Bias-Variance Reduction

Ensemble models help mitigate the errors caused by bias and variance in deep learning, resulting in bias-variance reduction. Bias refers to the discrepancy caused by approximating a complicated real-world situation with a simplified model, whereas variance pertains to the model’s susceptibility to minor fluctuations in the training data. Ensemble approaches, like bagging and boosting, mitigate bias by employing numerous weak learners and generate a superior and more easily interpretable model [22,52]. Such averaging can aid in lowering prediction variance, resulting in smoother predictions (that is, some models may have more fluctuating predictions, while other models may have more stabilizing predictions with the two impacts canceling each other out).

2.4.6. Dealing with Data Diversity

Ensemble models offer advantages when working with varied or dissimilar data. Various unique models can be trained on distinct subsets or representations of the data, capturing diverse elements or properties. By integrating these models, the ensemble can take advantage of the variety of information and generate more enlightened predictions. Ensemble approaches are capable of managing data that have varying scales, distributions, or modalities. This is because each individual model within the ensemble can focus on a single property of the data [53].

2.4.7. Flexibility and Modularity

Ensemble models possess a high degree of flexibility and modularity in their design and application. The models may be the result of the creation of a hybrid of algorithm models, such as decision trees, neural networks, or SVMs. This provides the ability to combine expertise in a certain field and utilize the advantages of unique algorithms. They may be readily expanded with more ensemble techniques as they develop and diverge, allowing for the modular inclusion and removal of individual models to accommodate new requirements or data [47].

2.4.8. Parallel Processing

Ensemble models may benefit from the parallel processing capabilities that are accessible. Training and prediction for individual models can be performed independently and simultaneously utilizing several processors or a distributed computational resource. This can lead to reduced training time and improved scalability, particularly when dealing with huge datasets or models [18].

2.4.9. Interpretability and Insights

Ensemble models lack interpretability on their own but can offer insights by assessing the influence and importance of individual models [23]. By analyzing the models or features that have the most impact on the predictions of the ensemble, analysts can gain a deeper understanding of the fundamental patterns and relationships within the data. Ensemble methods can be combined with interpretable models to provide the advantages of both high performance and interpretability [54].

2.4.10. Robustness to Concept Drift

Ensemble models have greater resilience to idea drift, which refers to the alteration of the statistical characteristics of the target variable over time. Ensemble approaches can effectively handle changes in data distributions by incorporating models trained on different time periods or utilizing alternative algorithms. Unlike fixed models that require periodic tuning, ensemble models can be updated dynamically. They can also be weighted to prioritize the most essential models based on incoming data, making them robust in non-stationary situations [51,55].

2.5. Block Diagram

In the process of synthesizing the required architecture, transfer learning was performed, which incorporated MobileNetV2 and ResNet50V2 models as feature extractors. Transfer learning is the idea of using features learned in one task to another different but related task, which helps the model to obtain benefits of pre-learned representations, and thus, the training from scratch is not always necessary. Figure 1 illustrates the development process for the proposed architecture from the preprocessing phase up to inferencing, as shown in Figure 1 below.

The algorithm used in SynthSecureNet is summarized in the above block diagram. From the input videos in the preprocessing phase, frames are extracted and augmented further with techniques that include flipping randomly, zooming randomly, adjusting the brightness, and rotating randomly to make the training dataset more diverse. During the feature extraction, the preprocessed frames are then analyzed using MobileNetV2 and ResNet50V2 models, respectively, which are used for feature extraction. MobileNetV2 and ResNet50V2 are convolutional neural networks (CNNs) that have been pre-trained on large-scale image databases including ImageNet and are capable of recognizing quality features within Images. To achieve this, the last convolutional layers of both MobileNetV2 and ResNet50V2 architectures are fed with the input frames to generate their feature maps, which preserves more high-level representations. In this last stage of the network (feature fusion phase), MobileNetV2 and ResNet50V2 feature maps are concatenated along with the channel dimension. Temporal concatenation integrates the two models by merging the feature maps along the channel axis, outputting a fused feature map that includes information from the two models. This fusion step enables the subsequent layers to make use of the complementary features being learned by both models.

In the Additional Layers phase, as shown in Figure 1 above, the fused feature map is for further processing of getting features and making predictions. These layers typically include a flattened layer that flattens the fused feature map obtained from the 3D convolution into a one-dimensional vector. Dense layers are fully connected layers that specifically calculate complex combinations of features that are extracted out of the flattened structure. Dropout layers come into play during the training process, where a certain part of the units is randomly removed in order to reduce overfitting by the use of this technique.

In the output layer phase, the final dense layer employs the sigmoid activation function in classifying the inputs as a binary output to signify the presence or the absence of violence in the input frame.

During the training phase, the binary cross-entropy function is used as the loss function for the SynthSecureNet model as it calculates the difference between the probability estimates produced by the model and the ground truth. The model is trained using the Adam optimizer because it allows for the learning rate to be updated for each parameter on the basis of its gradient history. As a post-processing of the individual frames of a video during inference, the preprocessed frames are input to the SynthSecureNet model for frame-level classification. To this end, the frame-level predictions are then combined based on methods such as majority voting or averaging to arrive at the final video-level prediction.

Furthermore, together with transfer learning, feature fusion, and additional layers, the proposed SynthSecureNet was designed to learn and classify violence in videos. The pre-trained models, when taken as feature extractors, are already powerful, as they employ and build from prior experience with large datasets of images. Incorporating features of multiple models promotes the models’ ability to focus on different and complementary aspects. These added layers are used in order to gain feature combinations and use them in training the model to learn accurate patterns and make precise predictions based on the fused features.

2.6. Model Training Dataset

The model was trained using the Real-Life Violence Situations Dataset during the training procedure. The data comprise clips that were captured in real-world scenarios, such as sports, as well as CCTV footage. The dataset is accessible online and is available for use [56]. The dataset comprises 2000 videos, of which 1000 videos depict violent activities, and the remaining 1000 videos depict non-violent activities. Only 70% of the videos from the Real-Life Violence Situations Dataset were allocated to the training set. Only 15% of the videos from the Real-Life Violence Situations Dataset were allocated to the validation set. The testing set was allocated 15% of the videos from the Real-Life Violence Situations Dataset.

The Real-Life Violence Situations Dataset, used for training and testing in this study, provides a balanced representation of violent and non-violent activities in real-world scenarios. However, it does not encompass all possible contexts of violence, such as variations in cultural environments, lighting conditions (e.g., low-light settings). These factors may significantly influence the performance of violence detection models in diverse applications.

In addition to the Real-Life Violence Situations Dataset, other datasets such as the Surveillance Camera Fight dataset [57], RWF-2000 [58], and Bus Violence dataset [59] offer diverse scenarios and environmental conditions. While these datasets could provide additional insights into violence detection across varied contexts, their inclusion was outside the scope of this study due to resource constraints and the focus on establishing the efficacy of SynthSecureNet on the Real-Life Violence Situations Dataset. Future work will incorporate these datasets to further evaluate and generalize the proposed approach.

3. Experimental Results

In this section, the performances of three architectures, namely, SynthSecureNet, ResNetV2, and MobileNetV2 Architectures, are compared.

3.1. Performance Metrics

The performances of these architectures are evaluated using five parameters, which include accuracy, recall, F1 score, precision, and training duration. The confusion matrix was also used and produced for each performance evaluation. The evaluation was in terms of accuracy, precision, recall, and F1 score [60]. Accuracy was used to evaluate overall correctness for a model. These factors that were used to evaluate performance are calculated using TP (true positive), FP (false positive), FN (false negative), and TN (true negative). Violence is a binary classification task where an event or human activity is determined as violent (positive class) or nonviolent (negative class) [61].

Accuracy is a measure of how many predictions are correct out of the total predictions. It is calculated as follows:

$$\mathrm{Accuracy}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$$

(17)

Accuracy provides a general idea of the correctness of the trained model. Precision, commonly referred to as Positive Predictive Value, indicates the percentage of the model’s positive predictions that turn out to be true. The formula to generate the precision is as follows:

$$\mathrm{Precision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(18)

Recall, which is a measure of how many actual positive cases the model accurately predicted, is sometimes referred to as Sensitivity or True Positive Rate. It is computed as follows:

$$\mathrm{Recall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(19)

The harmonic mean of recall and precision is known as F1 score. It offers a fair assessment that considers both false positives and false negatives. The formula for calculating the F1 score is as follows:

$$\mathrm{F}1\mathrm{Score}=2\times {\displaystyle \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(20)

Recall and precision are weighed equally in the F1 score [60].

In summary, accuracy gives an overall idea of how correct the model is during classification. Precision focuses on minimizing false positives. Recall focuses on minimizing false negatives. The F1 score balances precision and recall.

3.2. Experimental Procedure

The performance evaluation was performed by changing the size of the dataset that was used to train all the models. The datasets that were used to evaluate the all performances of the three models consist of a 400-video dataset, 800-video dataset, 1200-video dataset, 1600-video dataset, and 2000-video dataset, respectively. Across all sub-dataset sizes, the sets were allocated as follows: 70% for training set, 15% for validation set, and 15% for testing set.

All these datasets were extracted from one dataset that consists of 2000 videos in total. The dataset consists of two classes with 1000 videos each; the first class consists of violence videos, and the second class constitutes nonviolence videos. Sub-datasets were extracted evenly from both classes (e.g., 1200-video dataset = 600 violence videos + 600 nonviolence videos) of which 70%, 15%, and 15% were allocated to the training set, validation set, testing set, respectively. An online open and free programming platform called Kaggle Notebook (Python language) was used when developing the proposed new architecture and generating results for performance evaluation.

3.3. Performance Evaluation

3.3.1. Dataset of 400 Videos

In

Table 1. Performance comparison using the 400-video dataset.

Model	F1 Score	Recall	Precision	Accuracy	Training Duration
SynthSecureNet	96%	96%	96%	96%	15 min
ResNetV2	91%	88%	89%	89%	9 min
MobileNetV2	89%	89%	88%	88%	5 min

, a 400-video dataset was used to train, validate, and test the models. It can be observed in the table that SynthSecureNet outperformed both MobileNetV2 and ResNet50V2. Results in Table 1 were obtained during the testing phase, and the training duration that was observed during the training phase is also included in Table 1.

The SynthSecureNet architecture achieved an F1 score of 96%, a recall of 96%, a precision of 96%, and an accuracy of 96%, demonstrating a high level of performance for this model in violence detection. On the other hand, ResNetV2 achieved an F1 score of 91%, while MobileNetV2 achieved an F1 score of 89%, which shows that ResNetV2 outperformed MobileNetV2.

The training duration of the SynthSecureNet model (15 min) took longer than that of ResNet50V2 (9 min) and MobileNetV2 (5 min). When compared with both MobileNetV2 and ResNet50V2, the SynthSecureNet model’s high performance came at a trade-off of a small increase in training period.

Figure 2 shows the confusion matrices obtained after testing. SynthSecureNet made 2253 correct predictions and only 53 wrong predictions, demonstrating its effectiveness in accurately classifying violent and non-violent videos.

3.3.2. Dataset of 800 Videos

The models were trained, validated, and tested using a dataset of 800 videos, as shown in

Table 2. Performance comparison using the 800-video dataset.

Model	F1 Score	Recall	Precision	Accuracy	Training Duration
SynthSecureNet	98%	98%	98%	98%	30 min
ResNetV2	90%	88%	92%	89%	14 min
MobileNetV2	89%	89%	89%	89%	9 min

. SynthSecureNet performed better than both MobileNetV2 and ResNet50V2. The results in Table 2 were obtained during the testing phase, and the training duration that was observed during the training phase is also included in Table 2.

The SynthSecureNet architecture demonstrated a high degree of performance in violence detection with an F1 score of 98%, a recall of 98%, a precision of 98%, and an accuracy of 98%. However, ResNetV2 outperformed MobileNetV2 with an F1 score of 90% compared with 89% for MobileNetV2.

The training duration of the SynthSecureNet model, which lasted for 30 min, exceeded that of the ResNet50V2 model, which took 14 min, and the MobileNetV2 model, which took 9 min.

Figure 3 displays the confusion matrices acquired throughout the testing process. SynthSecureNet achieved a high accuracy in classifying between violent and non-violent activities, with 4567 right predictions and just 110 incorrect predictions.

3.3.3. Dataset of 1200 Videos

In

Table 3. Performance comparison using the 1200-video dataset.

Model	F1 Score	Recall	Precision	Accuracy	Training Duration
SynthSecureNet	98.31%	98.25%	98.37%	98.15%	46 min
ResNetV2	91%	89%	90%	90%	26 min
MobileNetV2	91%	92%	90%	90%	19 min

, the 1200-video dataset was used to train, validate, and test the models. It can be observed that SynthSecureNet outperformed both MobileNetV2 and ResNet50V2. The results in Table 3 were obtained during the testing phase, and the training duration that was observed during the training phase is also included in Table 3.

The SynthSecureNet architecture achieved an F1 score of 98.31%, a recall of 98.25%, a precision of 98.37%, and an accuracy of 98.15%, demonstrating a high level of performance for this model in violence detection. On the other hand, both ResNetV2 and MobileNetV2 achieved an F1 score of 91%, indicating that the level of reliability of both models is the same.

The training duration of the SynthSecureNet model (46 min) took longer than that of ResNet50V2 (26 min) and MobileNetV2 (19 min).

Figure 4 shows the confusion matrices obtained in the testing process. SynthSecureNet made 7374 correct predictions and only 139 wrong predictions, demonstrating its effectiveness in accurately classifying violent and non-violent activities.

3.3.4. Dataset of 1600 Videos

The models were trained, validated, and tested using a 1600-video dataset, as indicated in

Table 4. Performance comparison using the 1600-video dataset.

Model	F1 Score	Recall	Precision	Accuracy	Training Duration
SynthSecureNet	98.88%	98.89%	98.87%	98.77%	1 h 40 min
ResNetV2	91%	90%	91%	91%	30 min
MobileNetV2	91%	91%	90%	90%	1 h 11 min

. The table illustrates that SynthSecureNet outperformed both ResNet50V2 and MobileNetV2. The results in Table 4 were obtained during the testing phase, and the training duration that was observed during the training phase is also included in Table 4.

The SynthSecureNet architecture demonstrated a high level of performance in the detection of violence, as demonstrated by its F1 score of 98.88%, recall of 98.89%, precision of 98.87%, and accuracy of 98.77%. However, ResNet50V2 and MobileNetV2 both achieved an F1 score of 91%, indicating that their reliability levels are equivalent.

The SynthSecureNet model required an extended training duration (1 h and 40 min) compared with ResNet50V2 (30 min) and MobileNetV2 (1 h and 11 min).

Figure 5 illustrates the confusion matrices acquired during the testing process. The effectiveness of SynthSecureNet in accurately classifying violent and non-violent activities was demonstrated by its 9896 correct predictions and 123 incorrect predictions.

3.3.5. Dataset of 2000 Videos

In

Table 5. Performance comparison using the 2000-video dataset.

Model	F1 Score	Recall	Precision	Accuracy	Training Duration
SynthSecureNet	99.24%	99.23%	99.26%	99.22%	6 h 5 min
ResNetV2	94%	95%	94%	94%	3 h 9 min
MobileNetV2	91%	91%	90%	90%	2 h 30 min

, the 2000-video dataset was used to train, validate, and test the models. It can again be observed that SynthSecureNet outperformed both MobileNetV2 and ResNet50V2. The results in Table 5 were obtained during the testing phase, and the training duration that was observed during the training phase is also included in Table 5.

The SynthSecureNet architecture achieved an F1 score of 99.24%, a recall of 99.23%, a precision of 99.26%, and an accuracy of 99.22%. On the other hand, ResNet50V2 achieved an F1 score of 94%, while MobileNetV2 achieved an F1 score of 91%, which means that ResNet50V2 is more reliable when compared with MobileNetV2.

The training duration of the SynthSecureNet model (6 h and 5 min) took longer than that of ResNet50V2 (3 h and 9 min) and MobileNetV2 (2 h and 30 min).

Figure 6 shows the confusion matrices obtained in the testing process of all three models. SynthSecureNet made 12,454 correct predictions and only 98 wrong predictions, demonstrating its effectiveness in accurately classifying violent and non-violent activities.

3.4. Summary

In conclusion, it can be observed that the proposed SynthSecureNet architecture yields better results as compared with ResNet50V2 and MobileNetV2 in the task of violence detection from videos. SynthSecureNet achieved higher F1 scores, recall rate, precision level, and accuracy in all the sizes of the datasets considered, indicating its efficiency in classifying violence accurately. Despite the fact that the training times were longer than those of the other models, SynthSecureNet presented excellent results that seem to be justified by extra computational time.

The analysis of the evaluation results shows that SynthSecureNet is a promising violence detection system and can be considered rather effective and accurate. Therefore, the capacity for scaling big data and maintaining high performance makes it a suitable solution for actual use. The viability of the architecture reflects its durability and adaptability, which may make it appropriate for use in different areas with violence identification as a priority, for example, video surveillance, content filtering, safety, and security.

The consistent superior performance of SynthSecureNet across various dataset sizes demonstrates its robustness and scalability. This is particularly important in real-world applications where the volume of data can vary significantly. The architecture’s ability to maintain high accuracy levels even with larger datasets suggests that it could be effectively deployed in large-scale surveillance systems without compromising on detection quality.

Moreover, the trade-off between increased training time and improved performance is a crucial consideration. While SynthSecureNet requires more computational resources during the training phase, the enhanced accuracy in violence detection could lead to more efficient and effective real-time monitoring systems. This could potentially result in faster response times to violent incidents and improved public safety outcomes.

The flexibility of SynthSecureNet, as evidenced by its performance across different dataset sizes, also indicates its potential for adaptation to various contexts and scenarios. This versatility could be valuable in different sectors such as law enforcement, private security, and content moderation platforms, where the nature and prevalence of violent content may vary.

In summary, the experimental results strongly support the efficacy of the SynthSecureNet architecture for violence detection tasks. Its consistent outperformance of established models like ResNet50V2 and MobileNetV2 positions it as a promising advancement in the field of automated violence detection. Future work could focus on further optimizing the training process to reduce computational time while maintaining or even improving the current high levels of accuracy.

4. Conclusions

In this study, a new architecture named SynthSecureNet, was proposed as a feature-level fused model of the MobileNetV2 and ResNet50V2 architectures, which outperforms the individual models across various datasets independently. The results of the comprehensive evaluation on datasets containing 400 to 2000 videos show that our proposed model improves the performance of the existing ones.

The applications of the proposed architecture are relevant to multiple domains, such as video surveillance, content moderation, and public safety or security settings. The high accuracy and reliability of SynthSecureNet can help in the automation of violence detection, which can minimize the load of moderators and help in taking necessary precautions.

In this paper, feature-level fusion is also highlighted as the most suitable method to improve violence detection performance in the study. Hence, SynthSecureNet integrates parts of the models from both MobileNetV2 and ResNet50V2, which in turn enhances its predictive accuracy and generalization capability.

Future work can include exploring possibilities of minimizing the training duration and computational resource consumption.