Automated Sorting System for Skeletal Deformities in Cultured Fishes ^†

Abstract

Anomaly occurrence is a constant worldwide problem in aquaculture and it raises economic and animal welfare issues. The early-stage removal of abnormal fish from the stocks is necessary, and the sorting process remains manual worldwide, causing a significant increase in personnel cost and delays in the production cycle. The purpose of this project is to develop an integrated automated system for the valid sorting of farmed fishes by removing these with shape or colour anomalies or skeletal deformities. The sorting will be based on vision analysis and shape pattern recognition techniques.

1. Introduction

The appearance of anomalies is common in the juvenile stages of the fish biological cycle and may include the abnormal shape of a body part, miscolouring, deficient fin growth, a problematic swim bladder, skeletal deformities, etc. Some examples are presented in Figure 1. Although fish farms aim to the retain of increased standards of fish wellbeing and disease prevention, anomalies still exist. There are multiple generative factors with a synergistic effect for anomaly occurrence in fry fish. Some of these are heredity, total fish treatment, high stress levels [1,2,3,4], malnutrition, disease [5,6,7,8,9], and an improper farming environment [10].

Fry fish with obvious anomalies may later present a slower growth rate, or die, or ultimately have a reduced commercial value or be discarded. In each case, anomalies financially impact fish farms negatively, so relevant actions are necessary. Early-stage fish sorting is considered a necessity for fish farms [11] and it also aligns with the concept of animal wellbeing [12]. The sorting procedure is proven to consume far fewer resources than using resources to let all the fish grow.

Sorting fish with anomalies remains a manual procedure worldwide and has many disadvantages. It is a labour-intensive procedure, as it demands workers that will observe and sort fish manually. Also, this type of fish-by-fish observing occurs under intense lighting conditions and is an arduous procedure to perform. Furthermore, the process needs experienced workers for effective sorting. The threshold between commercially severe and non-severe deformities is empirically and subjectively defined by workers, due to the lack of a precise quality scale to allow skeletal development to be connected to the external morphology of the fish in various growing phases. This clearly means that manual sorting can be subjective and depend on the experience and condition of the worker that performs it, and obviously this time-consuming procedure causes a bottleneck in the production cycle of a fish hatchery.

The proper time phase for a single sorting to proceed in a hatchery is when the critical body parts of fry fish have mostly developed and their size is adequate for workers to observe potential anomalies. A multi-stage sorting strategy in various fry fish growth-cycle phases could lead to earlier anomaly detection and reduce the required nutritional resources for hatcheries. As fish sorting is manual, a multi-stage procedure would significantly increase labour costs and is infeasible.

The purpose of this project is the development of an integrated automated system for the valid sorting of farmed fish in the early stages of the growth cycle. The sorting will be based on vision analysis and shape pattern recognition techniques. The practical implication of the system is an improvement in sorting procedure efficiency and a significant reduction in the required time, the labour intensity, and the cost of the process.

2. Methods

The system is initially designed to function as a collaborative robot (co-bot) that combines a configuration with a fish feeding system and a closed cabin with controlled lighting conditions and components that feed visual content to a classification process running on a computational system, using advanced intelligent machine learning algorithms. Under this configuration, the co-bot device relies on a user to perform tasks that are easily performed by human hands under the instruction of the user interface, which is presented on a monitor. The user accesses the fish handling plate through an opening hatch on the cabin surface. The complete process is presented in Figure 2. The practical implication of such a co–botic system that performs the sorting is a significant reduction in labour intensity due to less required time for fish observance and the elimination of human subjectivity.

The algorithms running on the computational system perform multiple tasks, such as the recognition of fish spots over the handling plate, the recognition of probable cases of fish that overlap so that the user is notified to move them apart, and the recognition of visual anomalies and skeletal deformities. The feeding system provides batches of fish and, after a processing procedure, the user is given instructions over the monitor, in order to perform various tasks.

The training of the specific machine learning algorithms, started by manually creating a dataset of multiple pictures of fish, under the instruction of fish-sorting workers, is presented in Figure 3. Experienced sorting workers played a significant role in helping our team to tag the acceptable fish and sort those with anomalies, which were also tagged, as rejectable.

Significant geometric and color differences were studied in selected areas of interest between acceptable and discarded fish, as presented in Figure 4, so that an efficient percentage of recognition was achieved.

3. Results

For one of the trials for the validation of the proper function of the prototype device, the following tests were conducted:

• Step 1: A batch of a random number of fish was acquired, and the fish were manually counted to achieve 1394 fish.
• Step 2: The batch was fed to the system, and 124 fish were classified as “discard”. Also, the total number of the fish was automatically counted to 1394, which was valid.
• Step 3: The fish that were classified as “discard” were manually separated and examined by an experienced worker, and two (2) were sorted as “acceptable”.
• Step 4: The initial batch was formed again by mixing fish and then was manually sorted by an experienced worker. The number of fish that were found to be non-acceptable was 138. This led to the conclusion that the device had an 88.4% recognition success rate.
• Step 5: The manually pre-sorted batch that contained 1256 acceptable fish was fed to the device, and three were classified as “discard”.
• Step 6: The manually pre-sorted batch that contained 138 non-acceptable fish was fed to the device, and 123 of them were classified as “discard”, which led to the conclusion of an 89.1% recognition success rate, which is slightly higher than that in the previous test.

4. Discussion and Conclusions

The main feature of the prototype device is the non-acceptable fish recognition. In the conducted function tests, the lower percentage of recognition rate was 88.4%, which is considered to be acceptable for a prototype device. The slight difference in the recognition percentage between two tests could occur for various reasons, which could be due to random and uncontrollable facts, such as a small difference in the lighting reflections on the fish. The future goals of the project team include the improvement of the system classification rate.

As a future work, it would be interesting to calculate a break-even point for the device classification success rate, over which the cost of farming non-acceptable fish that were misclassified overcomes the labor costs of manual fish sorting.