*Article* **Identification of Shark Species Based on Their Dry Dorsal Fins through Image Processing**

**Luis Alfredo Carrillo-Aguilar 1, Esperanza Guerra-Rosas 2, Josué Álvarez-Borrego 1,\*, Héctor Alonso Echavarría-Heras <sup>1</sup> and Sebastián Hernández-Muñóz 3,4**


**Abstract:** Shark populations worldwide have suffered a decline that has been primarily driven by overexploitation to meet the demand for meat, fins, and other products for human consumption. International agreements, such as CITES, are fundamental to regulating the international trade of shark specimens and/or products to ensure their survival. The present study suggests algorithms to identify the dry fins of 37 shark species participating in the shark fin trade from 14 countries, demonstrating high sensitivity and specificity of image processing. The first methodology used a non-linear composite filter using Fourier transform for each species, and we obtained 100% sensitivity and specificity. The second methodology was a neural network that achieved an efficiency of 90%. The neural network proved to be the most robust methodology because it supported lower-quality images (e.g., noise in the background); it can recognize shark fin images independent of rotation and scale, taking processing times in the order of a few seconds to identify an image from the dry shark fins. Thus, the implementation of this approach can support governments in complying with CITES regulations and in preventing illegal international trade.

**Keywords:** CITES; shark fins; image processing; Fourier transform; neural network; non-linear composite filter

#### **1. Introduction**

The increased human exploitation and habitat deterioration over the last half-century has decreased shark populations worldwide [1,2]. Consequently, more than one-third of chondrichthyan species (sharks, rays, and chimeras, hereafter referred to as 'sharks') are threatened with extinction due to a myriad of human-caused threats; however, observed population declines are driven primarily by overexploitation in largely unregulated and unmonitored target and bycatch fisheries worldwide [3]. A global catch assessment estimated that approximately 100 million sharks are caught annually worldwide, including illegal, unreported, and unregulated catch [4]. The reassessment of 1199 species by the International Union for Conservation of Nature (IUCN) Red List reveals that almost 400 chondrichthyan species are jeopardized with extinction [5].

International efforts to improve the management and conservation of sharks have focused on the use of multilateral environmental agreements, such as the Convention on International Trade in Endangered Species of Wild Fauna and Flora (CITES), to ensure that products derived from shark and ray species are traded legally and sustainably [6]. At present, CITES has listed 46 shark and ray species in the Appendices, and the participating

**Citation:** Carrillo-Aguilar, L.A.; Guerra-Rosas, E.; Álvarez-Borrego, J.; Echavarría-Heras, H.A.; Hernández-Muñóz, S. Identification of Shark Species Based on Their Dry Dorsal Fins through Image Processing. *Appl. Sci.* **2022**, *12*, 11646. https://doi.org/ 10.3390/app122211646

Academic Editors: Enjin Zhao, Hao Qin and Lin Mu

Received: 6 September 2022 Accepted: 15 November 2022 Published: 16 November 2022

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

184 countries worldwide should monitor and control trading of shark products to ensure sustainability, legality, and traceability from international trade operations [7].

Economic globalization and exploitation of sharks have strengthened the demand and supply of domestic and international markets for sharks and ray products (mainly meat and fins) [8]. Shark meat markets have remained stable over the last decade, with Brazil, Spain, Uruguay, and Italy accounting for 57% of the average global shark meat imports [9–11]. In contrast, Hong Kong and mainland China are major worldwide trade and consumption centers for seafood, where shark fins are considered a prized cultural treasure and luxury food items, such as sharkfin soup—which is served on formal and special occasions [12].

Unfortunately, international trade data for sharks and their derivative products are rarely collected at the species level, hampering the monitoring of shark species or their derivative parts, such as fins and meat [8]. This represents a major challenge for the implementation of effective monitoring, enforcement, and requirements of countries referred to as parties—in meeting their obligations under CITES [13].

To achieve compliance with domestic and international regulations for the shark fin trade, there are several accessible identification tools to aid in the implementation of CITES trade controls for listed species, both domestically and at various points along the supply chain (i.e., software iSharkFin version number 1.4, fin guides, and genetic approaches). First, the bioinformatics tool, iSharkFin, developed by FAO and the University of Vigo, was designed to identify 39 species from wet shark fins [14,15]; however, some limitations need to be considered when using this software, including the misidentification of CITESlisted species, particularly when dry fins are analyzed because of the discordance between iSharkFin results, visual diagnostic characteristics, and genetic identification. Currently, this is the only software that is working.

Second, several visual shark fin identification (ID) guides can provide users with a fast and cheap tool for the identification of unprocessed fins from CITES species based on the morphological characteristics of certain fin types, such as the shape and coloration patterns [16–20]; however, the effectiveness of fin ID guides is highly dependent on the training and expertise of users in identifying fins from morphologically distinct species, such as *Sphyrna lewini* and *S. zygaena* [20].

Lastly, advances in molecular approaches that are typically used for the identification of shark and ray species, or their derivative products, in markets have made them more accessible than ever before because these assays can be performed quickly in basic laboratories and are relatively inexpensive. Two widely used genetic tools used to identify body parts at the species level, such as meat and fins, are (a) DNA barcoding (using the COI or ND2 mitochondrial genes [21–26]) and (b) multiplex PCR assays based on the nuclear ribosomal DNA internal transcribed spacer (ITS2) [27–32]. Nevertheless, in Latin American countries, due to financial and logistical restrictions for molecular analysis—such as the salary for a technician—dedicated molecular labs, and validation of a genetic tool for law enforcement systems and courts, DNA techniques are implemented as workflows for domestic inspections from importation, exportation, and re-exportation. As a result, there is an urgent need for a robust tool that can aid in the identification of shark fins in CITES enforcement contexts.

Here, we provide computer techniques and digital correlation systems that offer an accuracy-based solution for image processing, because we can determine the object position to identify the problem image. This first model (non-linear composite filter) has been selfdeveloped and the second model (neural network using the Local Binary Pattern) is a Matlab tool.

Most filters do not function efficiently when the problem image has small distortions, different sizes, rotations, or illumination. Therefore, in recent years, numerous efforts have been made to develop distortion-invariant systems using linear and non-linear filters [33]. Correlation filters were used to identify different species. For example, ceratium was identified with 90% efficiency, independent of images with different rotation sizes [34]. Subsequently, different shrimp tissues were identified to detect hypodermal necrosis and hematopoietic infection virus (IHHN) [35].

In addition, three different approaches involving molecular, morphometric, and image processing were implemented to identify wet and dry dorsal fins in two CITES-listed species (Isurus oxyrinchus and Lamna nasus) and a blue shark (Prionace glauca) from the Chilean shark fin market. The results showed that morphometric analysis lacked the accuracy to discriminate among species, whereas DNA-based identification and image processing were 100% successful [9].

In this study, we used two different image-processing approaches: a non-linear composite filter using the Fourier transform and a neural network to identify the species of origin of 37 dry dorsal fins sourced from 14 countries using photos of the global shark fin trade.

#### **2. Methodology**

#### *2.1. General Information about the Image Database*

The database used in this project was shark fin photos from the international fin trade established in the project "Enhancing the morphological tools to identify illegal shark fins traded in central America" financed by the Shark Conservation Fund in 2008. Part of this project includes 1029 photos of dry dorsal fins from 37 commercially important shark species taken from 14 different countries: United States, Mexico, Belize, Guatemala, Costa Rica, El Salvador, Panamá, Colombia, Ecuador, Perú, Chile, South Africa, Hong Kong, and Fiji. The database includes two groups (CITES-listed and non-CITES-listed). The CITES-listed species are very important because most of the shark populations are in critical danger, however, there are shark species that are not CITES-listed but are as important as the ones who are CITES-listed; that is why we decided to merge the two groups. The dry shark fin database was identified. Figure 1 shows four different dry shark fin species. The first one corresponds to *Sphyrna lewini* (A)*,* the second to *Sphyrna zygaena* (B)*,* the third to *Carcharodon carcharias* (C), and the last to *Trianodon obesus* (D). The photos were classified by species because we are interested in the population aspects of sharks and rays, including the genetic diversity, connectivity, and morphological supporting tools that can prevent the illegal trafficking of shark products in international trade in Latin America. To validate the use of the algorithms, all the shark fin photos were previously visually identified by shark fin identification experts [18–20], based on their knowledge and published fin field guides, and in particular, the experience training international workshop for government agencies who enforce international trade regulations of CITES-listed species. We compared the photos using two approaches: (i) a non-linear composite filter using Fourier transform and (ii) a neural network applied to test species identification from the dry shark fins. The images can be rotated or scaled. The algorithms were realized in Matlab language.

**Figure 1.** Dry shark fins from the first dry fin shark species up to the last dry fin shark species. (**A**) is *Sphyrna lewini*, (**B**) is *Sphyrna zygaena*, (**C**) is *Carcharodon carcharias* and (**D**) is *Trianodon obesus*. Database without noise in the background.

We created two databases for the neural network. The first dataset includes 1029 dry dorsal fins with a white background (without noise) and the second dataset contains 4438 dry dorsal fins with noise in the background (random variation of brightness or color information in the background of an image) (Figure 2). We gathered the second dataset of 4438 by removing the background of the first dataset of 1029 photos.

**Figure 2.** Dry dorsal fins shark database with noise in the background. In the four images we can see different objects, lines, and colors, which may hinder correct identification.

#### *2.2. Non-Linear Compositive Filter*

In this section, we present a detailed description of the non-linear composite filters. Figure 3 shows the steps of the non-linear composite filter. In step (A), on the left, there is an input training set (information of the species we want to recognize), *In*, defined by:

$$I\_n = \{f\_1(\mathbf{x}, y), f\_2(\mathbf{x}, y), \dots, f\_n(\mathbf{x}, y)\} \tag{1}$$

where *fi*(*x*, *y*) for *i* = 1, 2, ... , *n* is a two-dimensional function that represents a digitalized dry shark fin image. In this step, we have n dry shark fin photos, where each one is represented with *fi*(*x*, *y*).

**Figure 3.** Steps to obtain the non-linear composite filter.

Then, the fast Fourier transform (FFT) was applied to each of the images of the dry shark fins, and because the FFT is a linear integral, we took the total FFT:

$$F(\mu, v) = \sum\_{i=1}^{n} F\_i(\mu, v),\tag{2}$$

where *Fi*(*u*, *v*) represents the Fourier transform for each image in the training set; however, *n* is the total of dry shark fins in this set, and *u* and *v* are the frequency components (Step B).

Furthermore, *F*(*u*, *v*) can be written like:

$$F(u,v) = |F|^k \exp(i\varphi) \tag{3}$$

where *k* is a non-linear operator (0 ≤ *k* < 1) and *ϕ* is the phase (Step C).

With the *k* value selected, we get a better signal in both images. In this case, we choose *k* = 0; for this reason, the non-linear composite filter was realized with filters of phase only. The same procedure is applied to the input images (Steps D, E, and F). The results obtained from both the training and input images were multiplied to obtain a correlation plane [36] (Step G). If we have a single peak in the correlation plane, it means that we get a correct identification.

#### *2.3. Neural Network*

The second methodology consists of a neural network [37]. We used the local binary pattern function to obtain a vector of 59 elements for each image [38]. This algorithm is a simple and efficient descriptor that describes the textures (edges, corners, spots, and flat regions), and it is invariant to rotation and scale [39]. Furthermore, the Levenberg– Marquardt method is used [40]. The neural network consists of the following steps.

The neurons are simple information processors. The output layer comprises neurons that receive signals from the environment (*x*1, *x*2, *x*3, ... , *x*59). In this case, the input layer was the texture vector of the image. The hidden layer has three elements (error, weight, and sigmoid function). The errors and weights were random values. The sigmoid function transforms negative values into 0 and positive values are represented by 1; it is one of the most widely used non-linear activation functions. The mathematical expression is as follows:

$$y = \frac{1}{1 + \varepsilon^{-x}} \tag{4}$$

where *y*, (0 ≤ *y* ≤ 1) is the output and x is the real input value in the sigmoid function (logistic function).

The output layer consisted of 38 neurons, and each neuron was a dry shark dorsal fin. Figure 4 shows the steps of a neural network with one hidden layer, which are

described below. A three-layer neural network was used in this study. The first layer is an input layer containing 59 elements. The hidden layer was a single layer with 20 neurons, and the third layer was the output layer with 38 output values. Each of these outputs corresponds to a species of dry shark dorsal fin. Each of these features was assigned a random weight and an error value. In the hidden layer, the weight values are summed and the error is subtracted. The obtained value was affected by the sigmoid function. This procedure was performed to obtain the value in the output layer.

Of the 38 outputs, 37 belonged to each shark species studied in this study and one control group. This control group was created such that when a dry dorsal fin was identified and did not belong to any of the 37 species, the network would place it in the control group and thus avoid a possible error when identifying it with another species. These neural network steps are repeated as a cycle. In each neural network, 80% of the images were randomly selected for training, 10% were randomly selected for testing, and 10% were randomly selected for validating data. The photos of fins with different backgrounds that were used in the training of the neural network are not the same as those used to perform the validation and testing of the network. This procedure is performed until the global minimum value of the error function is obtained. The purpose of testing is to compare the outputs from the neural network against targets in an independent set, and the purpose of the validation set is to fine-tune the hyperparameters of the model and is considered a part of the training of the model [41]. Generally, 80% are used for training, 10% are used for testing, and 10% are used for validation in neural networks.

**Figure 4.** Scheme of a neural network with a hidden layer.

Finally, the percentages for sensitivity and specificity were applied to each result to determine the effectiveness of each methodology.

Sensitivity was defined as the proportion of individuals correctly identified as belonging to Species 1. The mathematical expression is as follows:

$$\frac{TP}{TP + TN} \tag{5}$$

where *TP* corresponds to true positives and *TN* corresponds to true negatives.

Specificity was defined as the proportion of correctly identified individuals that did not belong to Species 1.

$$\frac{TN}{TN + FP} \tag{6}$$

where *TN* corresponds to true negatives and *FP* corresponds to false positives.

#### **3. Results**

#### *3.1. Non-Linear Composite Filter*

The numerical simulations performed for the non-linear composite filters provided the most representative results for identifying dry shark fins from the CITES-listed and nonlisted species (n = 37). Table 1 shows that the species-specific composite filters developed for the 37 shark species showed excellent identification of CITES-listed and non-listed species (n = 37), with 100% sensitivity and specificity. In addition, the optimal value of the non-linear operator (k) was found to be 0.


**Table 1.** Sensitivity and specificity percentage of each of the dry dorsal fin species of sharks using the non-linear composite filter.

#### *3.2. Neural Network*

Four experiments were conducted using a neural network that varied the number of neurons in the hidden layer, species, and noise in the images. The second experiment was the best neural network because we obtained a 90% efficiency with 20 neurons in the hidden layer. Efficiencies between 84% and 88% were obtained in the other runs. The experiments were conducted as follows.

Table 2 shows the results from the first experiment with 37 shark species with a white background and one control group using ten neurons in the hidden layer. We repeated the neural network 15 times to determine the optimal neural network efficiency. The epochs are the number of cycles that the neural network performed to reach the global minimum value of the error function. Efficiency is a relative value that shows the ratio between the achieved result and the used resource. In this experiment, the best neural network achieved an efficiency of 88.9%.


**Table 2.** First experiment. Fifteen runs of the neural network with 37 species, 1 control group (38 "species"), and 10 neurons in the hidden layer.

Table 3 shows the second experiment with 37 shark species with a white background and one control group. We obtained an efficiency of 90% (shown in yellow) for the four neural networks.

**Table 3.** Second experiment. Fifteen runs of the neural network with 37 species, 1 control group (38 "species"), and 20 neurons in the hidden layer.


Table 4 shows the sensitivity and specificity percentage of each dry dorsal fin shark species using the neural network with 90% efficiency. In this table, we show the 100% sensitivity for *Carcharhinus plumbeus, Ginglymostoma cirratum* and *Negaprion acutidens*. *Carcharhinus limbatus* had a sensitivity of 56.43%; this was the lowest percentage of all species. The rest had between 65.34% and 99.04% sensitivity. The specificity was between 98.05% and 100%.

**Table 4.** Sensitivity and specificity percentage of each dry dorsal fin shark species using the neural network with 90% efficiency.


We performed a third experiment based on the first two experiments. Nine shark species had only five images, which is why they were not considered in this experiment. There were 27 dry shark fin species with a white background and one control group.

Table 5 shows the results of the third experiment, with 26 species and one control group. Here, we have three neural networks with an 89% efficiency.


**Table 5.** Third experiment. Fifteen runs of the neural network with 27 species and 20 neurons in the hidden layer.

Table 6 shows the fourth experiment with 37 species and one control group. Here, we have two neural networks with an 89% efficiency. In this experiment, there was a good percentage because the number of dry shark fins increased for each species.

**Table 6.** Fourth experiment. Fifteen runs of the neural network with 37 species, 1 control group (38 "species"), and 20 neurons in the hidden layer.


The final experiment (Table 6) was performed using a database of dry shark fin images with and without background noise to increase the number of dry fins in each species. From this database, 4438 images of the dry dorsal fins of sharks were obtained.

Table 7 shows the sensitivity and specificity of each dry dorsal fin shark species using a neural network with 89% efficiency. In this table, we show the 100% sensitivity for *Carcharhinus plumbeus*. *Sphyrna zygaena* had a sensitivity of 66.37%; this was the lowest percentage of all species. The rest had between 75% and 99% sensitivity. The specificity was between 98% and 99%.

**Table 7.** Sensitivity and specificity percentage of each dry dorsal fin shark species using the neural network with 89% efficiency. This table represents the database of dry shark fins with noise in the background of the images.


#### **4. Discussion**

The results obtained in this study show that the non-linear composite phase filter can successfully correlate (100%) with 37 different species of dry shark dorsal fins. In this context, the results obtained were similar to those obtained using species-specific composite filters to identify the dry fins (dorsal fins, right-sided pectoral fins, and caudal fins) of three shark species: *Prionace glauca, Isurus oxyrinchus*, and *Lamna nasus*. A 100% identification was recorded among the fins of each species analyzed [36]; however, in the study of [36], an inverse Gaussian filter was used to enhance the high frequencies, and the technique in [34] was used to have rotation invariance and the confidence level was calculated (95.4%). Only a phase filter was used in this study, and the percentages for the sensitivity and specificity were calculated.

A non-linear compound filter uses this value, *k,* as the non-linear operator. By changing the value to 1, we obtain a classically matched filter that has the advantage of optimizing the output when the input signal (image problem) is degraded by additive white noise [33]. When *k* = 0, we have a phase-only filter that maximizes the light efficiency in an optical system; moreover, when *k* = −1, we have an inverse filter that minimizes the correlation energy criteria. This last filter produces a narrower peak in the output correlation plane if the reference image and problem image are the same [34]. When the non-linear operator modifies the Fourier transform of the problem and reference images, we consider that we have a non-linear processor. The intermediate values of (0.1, 0.2, 0.3, ... , 0.9) allow us to vary the characteristics of the processor, such as the discrimination capacity and its variance to illumination [42]. It is essential to consider that in these results with the non-linear composite filter, the non-linear filter law (when *k* is different from zero) was discarded because when varying the value between 0 and 1, it was found that the best correlation peak was at *k* = 0. This represents a correlation using a phase-only filter [33].

The disadvantages of using the non-linear composite phase filter are as follows. Suppose we use n filters corresponding to n species. In this case, the algorithm takes a long time to process hundreds of problem images that contain different species; however, this disadvantage does not occur when using neural networks, since identifying a fin photo takes between 0.18 s to 0.48 s and information from all species has already been integrated.

The percentage of efficiency in the tables corresponds to the confusion matrix. The diagonal of this matrix indicates the number of fins correctly identified and the percentages outside of this diagonal shows all the fins that were not correctly identified.

The first neural network with a white background obtained 90% efficiency. *Sphyrna lewini* had 92% efficiency from 262 images, *Sphyrna zygaena* had 66% efficiency from 92 images, and *Sphyrna mokarran* had 78% efficiency from 16 images. In the second neural network with the noise in the background, we obtained 89% efficiency. *Sphyrna lewini* had 91% efficiency from 459 images, *Sphyrna zygaena* had 66% efficiency from 113 images, and *Sphyrna mokarran* had 82% efficiency from 104 images. This indicates that 66% of the images were correctly identified as belonging to *Sphyrna zygaena*. The low percentage of sensitivity is because there is less information from the images in both species; therefore, a more significant number of images is needed to obtain a more robust model. However, there are some species with 94–100% sensitivity, such as *G. unami*, *N. brevirostris*, and *N. acutidens,* which have high percentages because they do not look like the rest of the other species.

Having more variability in the database for each species will benefit the algorithm because it holds more information for each species and has a higher sensitivity percentage.

The local binary pattern function is essential because it is a texture classifier (that focuses on edges, corners, spots, and flat regions). It is designed to tolerate noise and handle grayscale, rotation, and scale-invariant images [38]. Therefore, our database is composed of photos in which some of the images are rotated in different directions to create a robust algorithm.

The advantage of using more than one layer and, in each layer, using more than 20 neurons is that we might obtain better efficiency, but, as a consequence, the neural network will take longer for training. Therefore, it will be better if we increase the number of images in each species to have better efficiency.

The neural network can be replicated to identify wet dorsal fins, as well as wet and dry pectoral fins. This is the first step in creating a tool for CITES agents to use to prevent international trade in the Asian market. Even so, building capacities for the implementation of CITES species is highly recommended in Latin American and global countries. Nevertheless, algorithmic tools must be provided to government agencies and inspectors in order to prevent international trade. Updating the identification of CITES species and non-CITES species with algorithms from machine learning systems could be salvageable in the future in order to conserve the remaining shark populations, which have been in decline since 1950 due to overfishing.

#### **5. Conclusions**

All species were identified using a non-linear composite filter with 100% sensitivity and specificity. Although a perfect percentage was obtained, this was not the best methodology for the following reasons: (1) It is not rotation- or scale-invariant; and (2) the filter takes a long time to identify a problem image (fin photo) because the problem image must be correlated with each image in the database. It can be made invariant if a non-linear composite phase filter is fed with hundreds of rotated and scaled images of the species to be identified. This filter can also be fed images with different illumination levels and fragmented images.

The best methodology for identifying dry dorsal fins for this study is the neural network, primarily because of the short time required to identify a species. The sensitivity and specificity of the studied species can be increased when the network is fed hundreds or thousands of images. Two high percentages were obtained in this study: 90% with images without a background and 89% with images with noise in the background. This methodology supports noisy images and is invariant to scale and rotation. In addition, it takes between 0.18 s and 0.48 s to identify a problem image (fin photo).

If we do not understand the problem impacting the shark populations in the following years, we would be responsible for driving all shark species to extinction because of a lack of conscience.

**Author Contributions:** Methodology, L.A.C.-A., H.A.E.-H. and S.H.-M.; Software, E.G.-R.; Validation, L.A.C.-A.; Formal analysis, J.Á.-B.; Investigation, L.A.C.-A., E.G.-R. and H.A.E.-H.; Data curation, S.H.-M.; Visualization, E.G.-R.; Supervision, J.Á.-B.; Project administration, S.H.-M.; Funding acquisition, J.Á.-B. and H.A.E.-H. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Centro de Investigación Científica y de Educación Superior de Ensenada (CICESE), Baja California, grant number F0F181.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Acknowledgments:** Luis Alfredo Carrillo Aguilar hold a Master degree in Centro de Investigación Científica y de Educación Superior de Ensenada (CICESE) supported by CONACYT scholarship. SHARK CONSERVATION FUNDING We thank Debra Abercrombie for reviewing this article.

**Conflicts of Interest:** The authors declare no conflict of interest related to this study.

#### **References**

