A Deep CNN-Based Salinity and Freshwater Fish Identification and Classification Using Deep Learning and Machine Learning

Abstract

Concerning the oversight and safeguarding of aquatic environments, it is necessary to ascertain the quantity of fish, their size, and their distribution. Many deep learning (DL), artificial intelligence (AI), and machine learning (ML) techniques have been developed to oversee and safeguard the fish species. Still, all the previous work had some limitations, such as a limited dataset, only binary class categorization, only employing one technique (ML/DL), etc. Therefore, in the proposed work, the authors develop an architecture that will eliminate all the limitations. Both DL and ML techniques were used in the suggested framework to identify and categorize multiple classes of the salinity and freshwater fish species. Two different datasets of fish images with thirteen fish species were employed in the current research. Seven CNN architectures were implemented to find out the important features of the fish images. Then, seven ML classifiers were utilized in the suggested work to identify the binary class (freshwater and salinity) of fish species. Following that, the multiclass classification of thirteen fish species was evaluated through the ML algorithms, where the present model diagnosed the freshwater or salinity fish in the specific fish species. To achieve the primary goals of the proposed study, several assessments of the experimental data are provided. The results of the investigation indicated that DenseNet121, EfficientNetB0, ResNet50, VGG16, and VGG19 architectures of the CNN with SVC ML technique achieved 100% accuracy, F1-score, precision, and recall for binary classification (freshwater/salinity) of fish images. Additionally, the ResNet50 architecture of the CNN with SVC ML technique achieved 98.06% and 100% accuracy for multiclass classification (freshwater and salinity fish species) of fish images. However, the proposed pipeline can be very effective in sustainable fish management in fish identification and classification.

1. Introduction

Bangladesh (BD) is an east–west–north–south riverine nation with a complex waterway network. Fish and fisheries are essential to Bangladeshi culture, economy, employment, and nutrition. Apart from rice, Bangladesh relies on fish. According to the Food and Agriculture Research Service (FRSS), the nation produced 961,458 metric tons of freshwater fish in 2014. The average daily fish consumption in the country is 52 g, although the recommended weekly amount is 68 g, according to the Department of Food (2014) [1,2]. BD residents receive 60% of their protein from fish. Aquaculture production increased from 712,640 to 2,060,408 metric tons between 2000 and 2016, surpassing wild capture production of 1.023 million tons. Fishing accounts for 4.37% of the country’s GDP and 23.37% of the agricultural sector. Fish exports make for 2% of the country’s foreign trade gains, at BDT 43,126.1 million, according to the Department of Fisheries (2014). Bangladesh has 1.2 million permanent fishermen. Seasonal fishing provides 10 million fishermen with cash or food for their families. The 2014–2015 Bangladeshi fiscal year saw 3,684,245 metric tons of fish production. This total included 1,023,991 metric tons (27.79%) from open inland waters, 2,060,408 from confined inland waters, and 599,846 from sea fishing. Fish live in freshwater, estuarine, or saline environments. Bangladesh is predicted to have 795 native fish and prawn species [1,2,3,4]. Besides, in the southernmost region of the Bay of Bengal, Bangladesh is blessed with abundant aquatic and coastal assets. As a result, it is ranked fifth in aquaculture and third in freshwater fish capture, according to the Food and Agriculture Organization of the United Nations. With an area of 4.73 million hectares (ha), the rivers and lakes of the southern nation of Bangladesh are home to about 2265 different species of fish from freshwater [5,6].

Punti, Anju, Artamim, Botika, Arwari, Bele, Baim, Kholshe, Boumach, Pabda, Chapila, Bilchuri, Meni, Darkina, and Murari are significant species of freshwater fish found in Bangladesh. In contrast, saltwater fish are native to the ocean, where they are appropriately named due to the presence of salt in the water surrounding them. Saltwater fish can live in a wide range of environments, including the icy waters of the Arctic, the warm waters of tropical seas, reefs of coral, saltwater ponds, mangrove forests, and the depths of the ocean [7,8]. The exact number of saltwater species of fish within Bangladesh is not accurately determined; however, it is widely reported in some published sources that there are approximately 475 saltwater fish species in the country. Recent research conducted by the Department of Fisheries (DoF) in Bangladesh included a listing of 343 different species of marine fish. The notable saltwater fish found in Bangladesh are Goldsilk seabream, Bengal yellowfin seabream, Spotted green goby, Banded eagle ray, Chacunda gizzard shad, etc. [5,9,10]. Nevertheless, the abundance of fish is diminishing due to the rising demand and consumption. By 2030, the world’s fish consumption is projected to reach 21.5 kg, up from 20.5 kg in 2018. In addition, the requirement for aquaculture is expanding in parallel with the expansion of the worldwide population, and it is anticipated that the total supply produced all over the world will increase by 62% by the year 2030 [11,12,13,14]. Also, the number of species of freshwater fish that are either already endangered or are in danger of becoming extinct in the coming decades is estimated to be twenty percent worldwide, and it is quite likely that this number will continue to rise in the years to come. In contrast, the eradication of certain fish species causes future generations to be ignorant of these species [6,15,16,17]. Thus, this is our key motivation for working on freshwater species and fish identification with cutting-edge technologies that allow people to quickly identify and learn about their types, species, habitats, and ecology.

On the other hand, machine learning (ML) and deep learning (DL) play a significant role in the field of automated identification and classification. DL and ML operate by training computers to identify patterns from enormous amounts of data. In fish identification, for instance, these systems examine several images of fish in order to learn to identify the distinctive features of each species. Once the training is completed, the system can immediately recognize a fish and classify it into the proper species only by scanning a fresh image. This is quite impactful; unlike hand fish identification, it saves time and effort. For researchers, environmentalists, and even those who like fishing, buying, and selling, accurate and fast identification aids in understanding and safeguarding marine species. Thus, this paper presents an intelligent technique to automatically classify fish into their respective categories of freshwater or saltwater, as well as identify the specific species of the fish. The contributions of this paper are as follows:

• The research focused primarily on collecting datasets on Bangladeshi freshwater and saltwater fish.
• The study outlines a multilevel classification pipeline using cutting-edge technologies, such as ML and DL. The first level of classification focuses on binary classification, determining whether a fish belongs to freshwater or salinity waters. After analyzing the fish categories, the system performs a multiclass classification in order to track the fish species.
• We also conducted numerous experiments on the trained models and enumerated the results accordingly.

There are five interconnected sections in this manuscript. Section 2 presents the background studies of related works. Section 3 provides the working procedure for the proposed method. Section 4 illustrates the experimental results, along with a comprehensive comparison. Finally, Section 5 presents the conclusion of this manuscript, along with existing drawbacks and future scopes.

2. Related Works

There have been numerous attempts to reliably recognize or identify fish species using deep learning (DL), artificial intelligence (AI), and machine learning (ML) techniques. The authors of [18] suggested an ML method that uses images to identify injuries and lice in real time in salmon farms. The researchers presented a 15-layer convolutional neural network (CNN) with 5 layers of dense structure for the identification of fish lice and wounds. In comparison to the well-established VGG-19 and VGG-16 models, which achieved accuracies of 91.2% and 92.8%, respectively, the suggested approach achieved a test accuracy of 96.7%. A survey of a cross-section of the scholarly literature on DL and ML’s practical uses in the fisheries and aquaculture sectors is presented in [19]. This encompasses a comprehensive examination of research and practical implementations in the following domains: (1) the aquaculture sector, including surveillance and oversight of the production atmosphere, the enhancement of supply use, and prevention of disease; (2) managing fisheries, where resource evaluation, fishing, capture surveillance, and supervision are integral components; (3) environmental monitoring, which pertains to hydrology, primary cultivation, and aquatic contamination, and (4) digitization of processing for fish, along with quality confirmation frameworks. Using DL techniques, the authors of [20] presented an automated computer vision system that is considered to be state-of-the-art for the categorization of fish species. Utilizing the characteristics generated by the suggested method, a support vector machine (SVM) was trained to classify test data. The outcome had an accuracy rate for classification of 94.3% for different kinds of fish. Automatic identification of adolescent American eels utilizing sonar data was achieved through the use of a DL method that employed a CNN in [21]. The authors of [22] set three standards for aquatic species identification techniques that are based on ecological study requirements: fish detection, functional characteristic forecasting, and fish categorization. These standards will help progress these systems through the use of state-of-the-art algorithms for DL. A classification accuracy of 61.38% and 54.80% was achieved by fish identification forecasting and functional characteristic identification models.

Using acoustic and ecological information, the authors of [23] demonstrated an ML algorithm for pelagic species school categorization. An accuracy of about 95% in identifying the fish species was attained using the suggested methods. The authors of [24] developed a new CNN model that uses DL techniques to categorize eight distinct types of fish. The researchers evaluated the suggested model (IsVoNet8) compared to ResNet50, VGG16, and ResNet101. Out of all the models that were compared, the one that came out on top was the IsVoNet8 approach, with a success accuracy of 98.62%. The objective of the study in [25] was to examine the immediate reaction of young gilthead seabream (Sparus aurata) when they were subjected to stress from excessive temperatures, high salinity, and ammonium. To identify Amazonian fish from images, the authors of [26] constructed a CNN and an image-filtering model (U-Net). An average accuracy of 97.9% was achieved by the developed CNN model in identifying 33 different fish species. To categorize eight distinct species of fish, the author of [27] designed and tested a fully automated real-time system utilizing Mask-RCNN and YOLO. AlexNet and ResNet-50 are the DL architectures that the researchers of [28] employed to categorize twenty native species of fish from freshwater. According to the findings of the study, the ResNet-50 model achieved the highest categorization accuracy of 100%. An automated technique for identifying and classifying fish species is described in [29]. Deep convolutional neural networks (DCNNs) provide the basis of the suggested model. It makes use of a simplified version of the AlexNet method, which is composed of two layers that are fully interconnected and four convolutional layers. A validation accuracy of 90.48% was attained by the suggested and altered AlexNet model with a reduced number of layers, in contrast to the primary AlexNet model’s performance of 86.65%.

To categorize the immature and adult trout, the authors of [30] focused on fish species classification and attempted to overcome the difficulties of categorizing extremely imbalanced data. Several DL architectures were tested in the study, including MobileNetV2, ResNet-50, MobileNetV1, and MobileNetV3. The findings showed that when comparing various models, MobileNetV1 always had the best accuracy. An automated method for identifying four commercially significant carp species was developed and implemented in [31] using a CNN-based VGG16 architecture. After evaluating the method using 5-fold cross-validation, the findings showed that it successfully identified four common carp varieties with a 100% accuracy rate. The authors of [32] detailed the process of creating FishDeTec, a smartphone app for recognizing freshwater fish native to Malaysia. Specifically, the VGG16 CNN model was utilized to construct the model for fish species detection. Artificial intelligence techniques, such as the K-Nearest Neighbor (KNN) algorithm, were used to detect and categorize freshwater fish species from photographs in [33]. Results showed that when tested on images of freshwater fish, the model achieved a performance level of 70% accuracy. To classify freshwater fish, the authors of [34] suggested a model that makes use of MobileNet V1 as an object detection method. The results demonstrated that the model achieved a 90% accuracy rate when it came to identifying different varieties of freshwater fish.

Furthermore, end-to-end approaches have led to the development of numerous DL techniques. Some authors have focused on feature extraction and then classified the extracted features based on their learning. Some of the authors first extracted the features from an image and then performed classification based on the extracted features. All of the previous studies not only classified the fish into binary classes (freshwater fish or salinity fish) but also worked on multiclass classification. Thus, we propose a multilevel classification technique, formerly known as detection and diagnosis, on fish image data using DL and ML approaches. The proposed method first extracts the significant features from a fish image. We then apply a series of classifiers to the extracted features, creating a multilevel classification pipeline. The proposed method is capable of identifying and categorizing fish species into freshwater and salinity categories. The current architecture utilizes two different types of datasets to identify and diagnose fish species. We assess the suggested system’s robustness, rapidity, and ability to identify and diagnose fish images, ensuring its practical application.

3. Materials and Method

This section is classified into two interconnected subsections. The Section 3.1 presents a description of the fish image dataset for both freshwater and salinity fish images. Then, the Section 3.2 presents the pipeline of the proposed method.

3.1. Dataset Description

Two different datasets were employed for the current framework. Firstly, the “BDFreshFish” dataset was used for freshwater fish species, and then the “fish-gres” dataset was utilized for salinity fish species [35,36]. The dataset “BDFreshFish” includes a collection of image data for eight distinct kinds of local freshwater fish that are found in different regions of Bangladesh. Batasio tengana, Anabas testudineus, Channa punctata, Heteropneustes fossilis, Marcrobrachium malcoimsonii, Mastacembelus armatus, Puntius sophore, and Ompok bimaculatus are the scientifically recognized names of the eight different types of fish for which images were collected in the dataset. The fish image collection consists of around 3100 photos distributed across 8 classes. On the other hand, the “fish-gres” dataset includes 8 different species of fish, with an average of 240–577 photos per species. The variation can be attributed to the accessibility of random samples collected from conventional markets situated in Gresik, East Java, Indonesia. Initially, the images that were acquired had an original dimension of 4160 × 3120 pixels; after that, the images were resized to 390 × 520 pixels. However, in the proposed work, only five species of salinity fish from the fish-gres dataset were used. The salinity fish species used in this work were Nibea Albiflora (252 images), Johnius Trachycephalus (240 images), Upeneus Moluccensis (577 images), Rastrelliger Faughni (544 images), and Eleutheronema Tetradactylum (240 images).

Table 1. The details of freshwater and salinity fish species datasets with sample data.

SL No.	Scientific Name of the Fish	Category		Local Name of the Fish	Number of Amassed Images	Sample Images
		Freshwater	Salinity
01.	Anabus testudineus	√		Koi	100
02.	Batasio tengana	√		Tengra	125
03.	Channa puctata	√		Taki	112
04.	Heteropneustes fossilis	√		Shing	102
05.	Marcobrachium malcoimsonii	√		Chingri	105
06.	Mstacemblelus armatus	√		Baim	105
07.	Ompok bimaculatus	√		Pabda	105
08.	Puntius	√		Puti	100
09.	Nibea albiflora		√	Poya Vola	252
10.	Johnius trachycephalus		√	Poya	240
11.	Upeneus moluccensis		√	Lal Koral	577
12.	Rastrelliger faughni		√	Macerel	544
13.	Eleutheronema tetradactylum		√	Tailla	240

displays the sample images of the freshwater and salinity fish species used in the proposed study. In this table, we have taken 8 species from the freshwater dataset and 5 species from the salinity water dataset. We selected five salinity fish species from the “fish-gres” dataset due to their significant availability in the Bangladeshi fish market. Other species in this dataset, however, are not available in the Bangladeshi fish market. The main goal of using two datasets was to build a multilevel classification model for fish identification and classification. Previous studies primarily used these datasets for multiclass classification to diagnose fish species separately. However, in our study, we integrated these two datasets to construct a model that utilizes machine learning and deep learning techniques for the classification of freshwater and salinity water types.

3.2. Proposed Method

The working operation of the current framework for the identification and classification of freshwater and salinity fish is presented in this section. In the current investigation, a total of thirteen distinct species of fish (among these, eight species were freshwater fish, and five species were saltwater fish) were taken into consideration. An architecture of interconnected AI models was proposed to recognize and categorize them. Choosing a class label for a new picture from among thirteen classes is more complicated than doing so with fewer classes, which is why multiclass classification is traditionally difficult. Another drawback of multiclass classifiers is their increased time complexity, and a further obstacle is the issue of data inequality. The model may learn to favor the most common classes and ignore the unusual ones when it is presented with a multiclass classification issue in which specific categories are uncommon while others are commonly seen. The last, but not least, important step is to choose a suitable model. Because each AI model is unique and subject to data- and task-specific constraints, there is no universally applicable approach. As a result, we conducted a comprehensive search to identify the most effective AI-enabled network for the classification of fish species from among the thirteen categories. Following an exhaustive process of data selection, network optimization, evaluation of models, network selection, and illustration, the chosen structure was implemented. Oversampling the photos of the categories with fewer photographs was performed to address data imbalance and bring the distribution of the groups into balance. To select the most suitable network, the researchers took into consideration the time complexity, accuracy, and compatibility of the systems. The accuracy of the framework was assessed through the implementation of cross-validation.

Figure 1 illustrates the overall operation of the suggested architecture. First, the fish images in this figure originated from Android phones, taken in nearby markets. Next, we applied image preprocessing techniques, such as image filtering, in order to remove the noise from the collected images. We used an image resizing technique to prepare the images for the next stage. Then, we utilized conventional pre-trained CNN models to extract the feature vectors from each individual image. We employed seven models, including DenseNet121, EfficientNetB0, InceptionV3, ResNet50, VGG16, VGG19, and Xception, on the preprocessed fish images to identify the feature vectors. Next, we stored the extracted features in CSV format and arranged the merged features based on the classes. We mainly focused on multilevel classification. We identified fish at the initial level, regardless of their freshwater or salinity; at the tertiary level, we combined the feature vectors based on the species stored in the dataset as individual classes. We then fed the feature vectors to conventional ML classifiers to determine the fish identification and classification outcomes. We used six ML classifiers (support vector classifier (SVC), random forest (RF), decision tree (DT), Gaussian naive Bayes (GNB), extreme gradient boosting (XGB), and logistic regression (LR)) to classify fish into binary categories (such as freshwater or salinity) by leveraging the best features from the CNN models. Finally, we employed the classifiers to diagnose the multiclass classification of freshwater fish (8 classes) and salinity fish (5 classes) across 13 fish species categories. Algorithm 1 demonstrates fish identification using both the ML and DL approaches.

Algorithm 1. Overall proposed pipeline and working principles

Input: Fish images captured via Android phones.

Output: Fish identification: freshwater/salinity water and species classification.

1.  BEGIN

2.      // Step 1: Image Acquisition

3.      Input: fish_images[] = Capture images using Android phones

4.

5.      // Step 2: Image Preprocessing

6.      FOR each image in fish_images[] DO

7.          preprocessed_image = ApplyFiltering(image)

8.           Load(preprocessed_image)

9.      END FOR

10.

11.      // Step 3: Feature Extraction using CNN Models

12.     features[] = []

13.     cnn_models[] = {DenseNet121, EfficientNetB0, InceptionV3, ResNet50, VGG16, VGG19, Xception}

14.

15.     FOR each model IN cnn_models[] DO

16.         FOR each preprocessed_image IN fish_images[] DO

17.             feature_vector = ExtractFeatures(model, preprocessed_image)

18.             Append(features[], feature_vector)

19.         END FOR

20.     END FOR

21.

22.     // Step 4: Store and Merge Features

23.     Save features[] AS CSV

24.      merged_features[] = MergeFeaturesByClass(features[])

25.

26.     // Step 5: Multilevel Classification

27.     // Level 1: Binary Classification (Freshwater/Salinity)

28.     binary_classifiers[] = {SVC, RF, DT, GNB, XGB, LR}

29.     binary_results[] = []

30.

31.     FOR each classifier IN binary_classifiers[] DO

32.         result = ClassifyBinary(classifier, merged_features[])

33.         Append(binary_results[], result)

34.     END FOR

35.

36.     // Level 2: Multiclass Classification (Freshwater: 08, Salty water: 05)

37.     multiclass_results[] = []

38.

39.     FOR each classifier IN binary_classifiers[] DO

40.         result = ClassifyMulticlass(classifier, merged_features[])

41.         Append(multiclass_results[], result)

42.     END FOR

43.

44.     // Step 6: Output Results

45.     Output: binary_results[], multiclass_results[]

46. END

4. Experimental Outcomes

This section presents the experimental result analysis of our proposed pipeline, along with the relevant discussion. First, this section provides an experimental result analysis of fish identification. Second, we present a multiclass classification of fish species. Finally, we illustrate a comprehensive comparative study of the existing method vs. our proposed model.

In this research, the authors investigated the effectiveness of seven pre-trained CNN models and seven ML techniques for recognizing and classifying images of fish. We ran all of the applications for this investigation on the Google Colab, which has 53 GB of RAM and a dedicated Graphics Processor Unit (GPU). The setup’s subscription type was ‘Pro subscription’. The pre-trained CNN extracted the features from a specific image and stored them for the classifiers to apply to the extracted features. For our investigation, we split the dataset into 80% training data and 20% testing data. To evaluate the efficacy of the proposed pipeline, we have enumerated the experimental results according to the model’s F1-score, accuracy, precision, and recall. We also performed the 05-fold cross-validation (k-fold cross-validation) technique in order to examine the efficacy of the proposed network.

4.1. Experimental Results on Fish Indentifcation

According to the proposed work, seven CNN-based frameworks were used to extract the significant features from the fish images. These features were then fed to seven machine-learning-based techniques to find the fish’s binary class—either freshwater or saltwater, with class 0 being freshwater and class 1 being saltwater.

Table 2. Results for binary classification with the CNN and ML techniques.

CNN-Based Feature Extractor	Classifiers	Avg. Accuracy (%)	Avg. Precision (%)	Avg. Recall (%)	Avg. F1-Score (%)
DenseNet121	SVC	1.0000	1.0000	1.0000	1.0000
	RF	1.0000	1.0000	1.0000	1.0000
	DT	0.9809	0.9824	0.9700	0.9759
	GNB	0.9656	0.9484	0.9698	0.9583
	XGB	1.0000	1.0000	1.0000	1.0000
	LR	1.0000	1.0000	1.0000	1.0000
EfficientNetB0	SVC	1.0000	1.0000	1.0000	1.0000
	RF	0.9943	0.9961	0.9897	0.9928
	DT	0.9503	0.9326	0.9466	0.9392
	GNB	0.9924	0.9926	0.9884	0.9905
	XGB	0.9981	0.9987	0.9966	0.9976
	LR	0.9981	0.9987	0.9966	0.9976
InceptionV3	SVC	0.9618	0.9580	0.9462	0.9519
	RF	0.9465	0.9594	0.9083	0.9298
	DT	0.8738	0.8406	0.8516	0.8458
	GNB	0.7610	0.7010	0.6580	0.6706
	XGB	0.9598	0.9462	0.9554	0.9506
	LR	0.9503	0.9326	0.9466	0.9392
ResNet50	SVC	1.0000	1.0000	1.0000	1.0000
	RF	1.0000	1.0000	1.0000	1.0000
	DT	0.9714	0.9677	0.9621	0.9648
	GNB	0.9942	0.9921	0.9940	0.9930
	XGB	0.9981	0.9987	0.9967	0.9977
	LR	1.0000	1.0000	1.0000	1.0000
VGG16	SVC	1.0000	1.0000	1.0000	1.0000
	RF	0.9943	0.9961	0.9897	0.9928
	DT	0.9484	0.9265	0.9516	0.9378
	GNB	0.9694	0.9586	0.9662	0.9623
	XGB	0.9962	0.9952	0.9952	0.9952
	LR	1.0000	1.0000	1.0000	1.0000
VGG19	SVC	1.0000	1.0000	1.0000	1.0000
	RF	0.9981	0.9987	0.9966	0.9976
	DT	0.969	0.9209	0.9227	0.9218
	GNB	0.9598	0.9462	0.9554	0.9506
	XGB	0.9981	0.9987	0.9966	0.9976
	LR	1.0000	1.0000	1.0000	1.0000
Xception	SVC	0.9895	0.9899	0.9816	0.9856
	RF	0.9885	0.9857	0.9857	0.9857
	DT	0.9063	0.8793	0.8930	0.8857
	GNB	0.8757	0.8536	0.8299	0.8404
	XGB	0.9924	0.9885	0.9926	0.9905
	LR	0.9885	0.9857	0.9857	0.9857

displays the experimental findings for the binary classification using CNN and ML techniques. According to Table 2, previously trained CNN models, such as DenseNet121, EfficientNetB0, ResNet50, VGG16, and VGG19, acquired the highest accuracy, F1-score, precision, and recall of 100% with the SVC ML classifier in binary classification (freshwater/salinity) of the fish images. On the other hand, InceptionV3 (96.18%) and Xception (99.24%) CNN models with SVC and XGB ML classifiers did not achieve benchmark accuracy in binary fish classification. Additionally, the comparison among the seven CNN architectures’ best results with the SVC and XGB ML classifiers and time complexity is shown in

Table 3. Comparison results for binary classification.

Techniques	Avg. Accuracy (%)	Avg. Precision (%)	Avg. Recall (%)	Avg. F1-Score (%)	Time Complexity
DenseNet121 + SVC	1.0000	1.0000	1.0000	1.0000	483 ms ± 11 ms
EfficientNetB0 + SVC	1.0000	1.0000	1.0000	1.0000	483 ms ± 9.39 ms
InceptionV3 + SVC	0.9618	0.9580	0.9462	0.9519	2.23 s ± 40.1 ms
ResNet50 + SVC	1.0000	1.0000	1.0000	1.0000	927 ms ± 8.72 ms
VGG16 + SVC	1.0000	1.0000	1.0000	1.0000	2.46 s ± 13.7 ms
VGG19 + SVC	1.0000	1.0000	1.0000	1.0000	2.39 s ± 24.7 ms
Xception + XGB	0.9924	0.9885	0.9926	0.9905	2.62 s ± 35.6 ms

. From this table, we can clearly observe that the DenseNet121+SVC pipeline provided optimal results in terms of accuracy, precision, recall, F1-score, and the time complexity of the technique. To measure the performance of the proposed technique, a learning curve, confusion matrix, and bar chart were utilized. Figure 2, Figure 3 and Figure 4 illustrate the present framework’s performance measurements of the learning curve, confusion matrix, and bar chart.

4.2. Multiclass Classification of Fish Species

4.2.1. Eight Freshwater Fish Classifications

To diagnose the multiclass classification of fish images, we implemented the seven ML-based algorithms to the binary classifier results to identify the eight freshwater fish species through the freshwater binary classification images. The results for the multiclass classification for freshwater fish with CNN and ML techniques are displayed in

Table 4. Results for multiclass classification (freshwater fish) with CNN and ML techniques.

CNN-Based Feature Extractor	Classifiers	Avg. Accuracy (%)	Avg. Precision (%)	Avg. Recall (%)	Avg. F1-Score (%)
DenseNet121	SVC	0.8025	0.8186	0.7994	0.7983
	RF	0.8790	0.8856	0.8756	0.8753
	DT	0.6943	0.7015	0.6870	0.6874
	GNB	0.6624	0.7398	0.6060	0.6454
	XGB	0.8917	0.89292	0.8839	0.8855
	LR	0.9108	0.9137	0.9067	0.9069
EfficientNetB0	SVC	0.9490	0.9490	0.9486	0.9481
	RF	0.9108	0.9090	0.9090	0.9079
	DT	0.6624	0.6616	0.6569	0.6552
	GNB	0.8599	0.8651	0.8582	0.8586
	XGB	0.8917	0.8912	0.8875	0.8881
	LR	0.9809	0.9799	0.9810	0.9798
InceptionV3	SVC	0.5669	0.5687	0.5577	0.5410
	RF	0.5987	0.5963	0.5931	0.5706
	DT	0.4331	0.4170	0.4287	0.4149
	GNB	0.4076	0.3322	0.3978	0.3461
	XGB	0.6369	0.6456	0.6343	0.6209
	LR	0.6306	0.6319	0.6252	0.6210
ResNet50	SVC	0.9806	0.9802	0.9813	0.9803
	RF	0.9209	9311	0.9305	0.9289
	DT	0.7097	0.7097	0.7132	0.7052
	GNB	0.9484	0.9521	0.9506	0.9499
	XGB	0.9419	0.9443	0.9441	0.9423
	LR	0.9677	0.9690	0.9710	0.9682
VGG16	SVC	0.9363	0.9385	0.9311	0.9335
	RF	0.9299	0.9260	0.9215	0.9235
	DT	0.6561	0.6851	0.6573	0.6633
	GNB	0.7389	0.8631	0.7430	0.7638
	XGB	0.9045	0.8997	0.8976	0.8978
	LR	0.9554	0.9549	0.9517	0.9529
VGG19	SVC	0.9745	0.9754	0.9762	0.9752
	RF	0.9363	0.9373	0.9354	0.9355
	DT	0.6688	0.6692	0.6604	0.6609
	GNB	0.7643	0.8470	0.7632	0.7762
	XGB	0.9236	0.9240	0.9209	0.9215
	LR	0.9745	0.9748	0.9738	0.9739
Xception	SVC	0.5096	0.4453	0.5027	0.4612
	RF	0.6561	0.6545	0.6522	0.6509
	DT	0.5223	0.5189	0.5133	0.5110
	GNB	0.5669	0.5876	0.5518	0.5399
	XGB	0.7197	0.7115	0.7146	0.7110
	LR	0.7261	0.7288	0.7227	0.7190

. According to Table 4, the previously trained CNN models, EfficientNetB0 and ResNet50, acquired the highest accuracy of 98.09% and 98.06% with LR and SVC ML techniques in the multiclass freshwater fish classification of the fish images from binary classification. However, DenseNet121 (91.08%), VGG16 (95.54), VGG19 (97.45%), Xception (72.61%), and InceptionV3 (63.69%) CNN models with SVC, LR, and XGB ML classifiers did not achieve benchmark accuracy in multiclass freshwater fish classification. Additionally, the comparison among the seven CNN algorithms’ best results with the SVC, LR, and XGB ML classifiers and time complexity is shown in

Table 5. Comparison results for multiclass classification (freshwater fish).

Techniques	Avg. Accuracy (%)	Avg. Precision (%)	Avg. Recall (%)	Avg. F1-Score (%)	Time Complexity	MACs	FLOPs
DenseNet121 + LR	0.9108	0.9137	0.9067	0.9069	245 ms ± 34.4 ms	4.144G	8.360G
EfficientNetB0 + LR	0.9809	0.9799	0.9810	0.9798	233 ms ± 36.6 ms	45.840M	89.789M
InceptionV3 + XGB	0.6369	0.6456	0.6343	0.6209	11.7 s ± 180 ms	5.840G	11.430G
ResNet50 + SVC	0.9806	0.9802	0.9813	0.9803	480 ms ± 1.46 ms	4.242G	8.301G
VGG16 + LR	0.9554	0.9549	0.9517	0.9529	1.06 s ± 57.4 ms	15.480G	30.945G
VGG19 + SVC	0.9745	0.9754	0.9762	0.9752	1.14 s ± 8.33 ms	19.482G	39.252G
Xception + LR	0.7261	0.7288	0.7227	0.7190	769 ms ± 216 ms	8.632G	16.871G

. Also, we interpreted the Multiply-Accumulates (MACs) and Floating-Point Operations (FLOPs) of the models, where we found the highest result from the hybrid models for freshwater fish classification (Table 5). To measure the performance of the proposed technique, the learning curve, confusion matrix, bar chart, and receiver operating characteristic curve (ROC curve) have been utilized. Figure 5, Figure 6, Figure 7 and Figure 8 illustrate the current framework’s performance measurements of the learning curve, confusion matrix, ROC curve, and bar chart.

4.2.2. Five Salinity Fish Classifications

To identify the multiclass salinity fish classification using the present framework, we utilized the seven ML-based models to diagnose the five salinity fish species multiclass classification using binary classification of salinity fish data. The experimental findings for the multiclass salinity fish classification with CNN and ML techniques are displayed in

Table 6. Results for multiclass classification (salinity fish) with CNN and ML techniques.

CNN-Based Feature Extractor	Classifiers	Avg. Accuracy (%)	Avg. Precision (%)	Avg. Recall (%)	Avg. F1-Score (%)
DenseNet121	SVC	0.7814	0.7665	0.7244	0.7324
	RF	0.9235	0.9205	0.9073	0.9131
	DT	0.7131	0.6843	0.7040	0.6897
	GNB	0.4836	0.5590	0.5707	0.4888
	XGB	0.9290	09244	0.9194	0.9211
	LR	0.9563	0.9599	0.9410	0.9490
EfficientNetB0	SVC	0.9809	0.9788	0.9788	0.9788
	RF	0.9672	0.9685	0.9577	0.9626
	DT	0.6776	0.6394	0.6271	0.6324
	GNB	0.8607	0.8370	0.8797	0.8503
	XGB	0.9563	0.9515	0.9526	0.9499
	LR	0.9891	0.9899	0.9871	0.9884
InceptionV3	SVC	0.6038	0.6085	0.5255	0.5360
	RF	0.6585	0.6607	0.6173	0.6287
	DT	0.4699	0.4634	0.4703	0.4635
	GNB	0.3880	0.4040	0.3773	0.3494
	XGB	0.7240	0.7118	0.6928	0.6994
	LR	0.7295	0.7365	0.7376	0.7343
ResNet50	SVC	1.0000	1.0000	1.0000	1.0000
	RF	0.9891	0.9899	0.9871	0.9884
	DT	0.8224	0.8108	0.8128	0.8116
	GNB	0.9372	0.9217	0.9547	0.9333
	XGB	0.9891	0.9904	0.9851	0.9876
	LR	0.9973	0.9983	0.9982	0.9982
VGG16	SVC	0.9945	0.9937	0.9964	0.9950
	RF	0.9863	0.9863	0.9836	0.9849
	DT	0.8279	0.7995	0.8114	0.8046
	GNB	0.8579	0.8723	0.8118	0.8334
	XGB	0.9863	0.9882	0.9882	0.9882
	LR	0.9945	0.9937	0.9964	0.9959
VGG19	SVC	0.9945	0.9937	0.9964	0.9950
	RF	0.9918	0.9918	0.9917	0.9917
	DT	0.8115	0.7927	0.7829	0.7869
	GNB	0.7896	0.8100	0.7468	0.7607
	XGB	0.9891	0.9851	0.9908	0.9878
	LR	0.9945	0.9937	0.9964	0.9950
Xception	SVC	0.5847	0.6143	0.5249	0.5215
	RF	0.7186	0.7426	0.7017	0.7167
	DT	0.5109	0.5089	0.5009	0.5025
	GNB	0.2923	0.3555	0.3706	0.2792
	XGB	0.7568	0.7583	0.7495	0.7494
	LR	0.7432	0.7420	0.7689	0.7507

. As seen in Table 6, the previously trained CNN model ResNet50 acquired the highest accuracy, F1-score, precision, and recall of 100% with the SVC ML technique in the multiclass salinity fish classification from the fish images of the binary salinity classification. On the other hand, DenseNet121 (95.63%), EfficientNetB0 (98.91%), VGG16 (99.45%), VGG19 (99.45%), Xception (75.68%), and InceptionV3 (72.95%) CNN models with SVC, LR, and XGB ML classifiers did not achieve benchmark accuracy in the multiclass salinity fish classification. Furthermore, the comparison among the seven CNN architectures’ best results with the SVC, LR, and XGB ML classifiers and time complexity are shown in

Table 7. Comparison results for multiclass classification (salinity fish).

Techniques	Avg. Accuracy (%)	Avg. Precision (%)	Avg. Recall (%)	Avg. F1-Score (%)	Time Complexity	MACs	FLOPs
DenseNet121 + LR	0.9563	0.9599	0.9410	0.9490	341 ms ± 17.6 ms	4.134G	8.267G
EfficientNetB0 + LR	0.9891	0.9899	0.9871	0.9884	324 ms ± 27 ms	44.840M	89.680M
InceptionV3 + LR	0.7295	0.7365	0.7376	0.7343	1.01 s ± 347 ms	5.749G	11.498G
ResNet50 + SVC	1.0000	1.0000	1.0000	1.0000	1.2 s ± 8.76 ms	4.134G	8.267G
VGG16 + LR	0.9945	0.9937	0.9964	0.9959	1.24 s ± 13.6 ms	15.470G	30.941G
VGG19 + SVC	0.9945	0.9937	0.9964	0.9950	2.78 s ± 54.9 ms	19.632G	39.264G
Xception + XGB	0.7568	0.7583	0.7495	0.7494	13.7 s ± 232 ms	8.442G	16.884G

. Also, we interpreted the MACs and FLOPs of the models, where we found the highest result from the hybrid models for the salinity water fish classification (Table 7). To measure the performance of the suggested technique, the learning curve, confusion matrix, and bar chart were utilized. Figure 9, Figure 10, Figure 11 and Figure 12 illustrate the present framework’s performance measurements of the learning curve, confusion matrix, and bar chart.

4.3. Comprehensive Analysis of the Current Work

The suggested strategy’s comparison to the previously developed methods is displayed in

Table 8. Comprehensive analysis of the suggested work.

Previous Works	Techniques Employed		Architectures/Frameworks	Used Dataset	Fish Class		Accuracy
	DL	ML			Freshwater	Salinity
[1]	√	×	VGG16, VGG19, abd CNN	Fish images were collected from fish tanks during October 2021 to January 2022.	×	√	96.7%
[3]	√	√	CNN and SVM	16 species of fish videos were collected from islands of Western Australia during 2011–2013.	×	√	94.3%
[4]	√	×	CNN	A laboratory was used to take images of American eels from tanks.	×	√	98%
[6]	√	×	Neural Network	The data were collected from surveys during 2005–2015 in the central Mediterranean Sea.	×	√	95%
[7]	√	×	ResNet50, ResNet101, and VGG16	A fish dataset of 8 salinity species of fish was used.	×	√	98.62%
[9]	√	×	U-Net and CNN	A total of 141 freshwater species of fish images were taken from Morona River.	√	×	97.9%
[10]	√	√	Mask-RCNN and YOLO	DIDSON dataset images of 8 freshwater fish species were captured from the Ocqueoc River.	√	×	66.9%
[11]	√	×	AlexNet, CNN, and ResNet-50	20 freshwater varieties of fish images were taken from the ponds of Assam.	√	×	100%
[12]	√	×	Deep CNN and AlexNet	QUT and LifeCled2015 freshwater fish datasets were used with 6 species.	√	×	90.48%
[13]	√	×	MobileNetV2, ResNet-50, MobileNetV1, and MobileNetV3	Images of mature and immature trout were taken from the rivers of Norway.	√	×	90%
[14]	√	×	VGG16 and CNN	4 species of freshwater fish images were used from a fish tank.	√	×	100%
[15]	√	×	VGG16 and Transfer Learning	8 freshwater varieties of fish images were taken from Malaysia.	√	×	N/A
[16]	×	√	K-Nearest Neighbor	10 freshwater varieties of fish images were taken from various sources.	√	×	70%
[17]	√	×	MobileNetV1	3 freshwater varieties of fish images were used.	√	×	90%
Proposed Framework	√	√	DenseNet121, EfficientNetB0, InceptionV3, ResNet50, VGG16, VGG19, Xception, SVC, RF, DT, GNB, XGB, and LR	The dataset “BDFreshFish” includes a collection of eight distinct kinds of local freshwater fish. The “fish-gres” dataset includes 8 different species of fish. Among these 8 varieties, the proposed system used 5 species.	√	√	100% (binary classification), 98.06%, and 100% (multiclass classification of freshwater and salinity fish species)

, including the method that is considered to be the state-of-the-art method, in order to guarantee that it is effective.

5. Conclusions

The recognition and categorization of varieties of fish is of considerable importance to marine scientists to study marine ecosystems and fish habits, as well as for the investigation of threatened species populations. Additionally, it is significant for the fishery and seafood business, which plays a role in the management of the fishing industry to create a large amount of income and employment, which is a big part of the world economy, as well as in Bangladesh. This is why we presented deep CNN with ML methods to classify and identify the fish species into binary classes (such as salinity/freshwater) and then diagnosed the fish images into thirteen multiclass classifications of fish species. Seven CNN frameworks were utilized to extract the important features from the images so that the seven ML techniques could utilize the features to identify the binary class (freshwater/salinity) of fish species. Additionally, the multiclass classification of thirteen fish species was assessed through the ML models. Many experimental operations were evaluated to acquire the objectives of the present research. The findings of the current work demonstrated that the previously trained DenseNet121, EfficientNetB0, ResNet50, VGG16, and VGG19 models of the CNN with the SVC ML technique achieved 100% accuracy, F1-score, precision, and recall for binary classification (freshwater/salinity) of fish images. Additionally, the ResNet50 architecture of the CNN with the SVC ML technique achieved 98.06% and 100% accuracy for multiclass classification (thirteen fish species of freshwater and salinity fish) of fish images.

While working with the proposed model and architecture, we discovered some existing drawbacks that could be improved in the future. Firstly, the proposed model was tested on the fish species in Bangladesh. Regions and locations can greatly influence the diversity and variety of fish species. For instance, we included Mstacemblelus armatus, also known as the water fish, in our study. Globally, this species is commonly referred to as an eel fish. If the model feeds on any unknown species of fish, it cannot be reliable enough to provide an accurate classification. Secondly, the image quality, background, and noise had a significant impact on the model. Thirdly, we did not employ the trained model in lightweight applications, such as Android and IOS. We will identify more datasets with a wider variety of fish species and effective deep learning techniques in the future, along with diagnosing fish images. We are planning to detect the misclassified samples that affected the accuracy of the models. We will enrich our dataset with a set of image preprocessing operations in order to reduce noise and enrich the lighting conditions. Additionally, we will create a lightweight mobile application to implement the YOLOV5 heavyweight model in real-life classification scenarios. However, this work contributed to sustainability by improving the accurate classification of fish species, which is crucial for the management of marine ecosystems and the advancement of sustainable fishing techniques. Additionally, it enhanced the economic viability of the fishing sector by facilitating improved resource management and technical progress.