**1. Introduction**

Increase in global food demand has led to livestock breeders seeking to produce breeding lines more able to match economic demand which have genetic advantages to primary traits such as growth speed and reduced feed intake. With agricultural animals providing 18% of global calories and 39% of global protein intake, they are still an essential part of global nutritional requirements [1]. One of the methods in making livestock more advantageous is to selectively breed them for commercial traits such as growth rate [2], milk quality [3], weather [4] and disease resistance [5]. Recent improvements in genomic technologies such as detection of single nucleotide polymorphisms (SNPs) and whole genome sequencing [6] have allowed unparalleled insight into the driving factors which guide animal phenotypes [7] and successful genomic breeding selection has been able to identify traits which are not only desirably economically, such as improved livestock social behaviour and carcass composition [5], but also identify novel cosmetic or welfare indicators such as predicting horn phenotypes in Merino sheep [8]. As the number and biological complexity of known phenotypes are increasing, there is a call to innovate new ways to detect phenotypes faster and more accurately [9] in addition to detecting and preserving those of potential future relevance [10].

Non-invasive imaging techniques, such as computed tomography (CT), magnetic resonance imaging (MRI) and ultrasound, can provide detailed data from which phenotypes can then be extracted [11,12] and used in breeding programmes. One major benefit

**Citation:** Robson, J.F.; Denholm, S.J.; Coffey, M. Automated Processing and Phenotype Extraction of Ovine Medical Images Using a Combined Generative Adversarial Network and Computer Vision Pipeline. *Sensors* **2021**, *21*, 7268. https://doi.org/ 10.3390/s21217268

Academic Editor: Sylvain Girard

Received: 16 August 2021 Accepted: 28 October 2021 Published: 31 October 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

of using these non-invasive imaging techniques is that internal phenotypic data, such as muscle and fat distribution [13], organ size and limb morphology, can then be incorporated more swiftly into genetic breeding programmes for live breeding animals [14]. Out of the commonly used non-invasive imaging techniques, CT scanning provides the highest resolution (1–2 mm). One hurdle which can impact extraction of useful phenotypic information is the processing and analysis of these images which can be time consuming and therefore costly, especially if there is a need to re-analyse historic databases to measure newly emerging phenotypes.

Machine learning and artificial intelligence have been successfully implemented to increase phenotype detection speed and accuracy within many different medical areas including brain cancer detection, COVID status in lungs and classification of organ deformities [15–17]. Recently the same technology has been applied to areas of agricultural science such as detection of bovine tuberculosis status based upon milk spectral data [18,19]. Briefly put, these networks work by passing data such as images, or segments thereof, through a series of layers containing artificial neurones which determine the likelihood of visually similar animals such as pigs, sheep or horses on a scale of 0 (absent) to 1 (present). The type of network commonly used to perform this image to binary diagnostic is a convolutional neural network where, as the layer depth increases, many datapoints (such as pixels) are condensed into fewer datapoints (likelihood of, e.g., pig, sheep or horse presence). The subject field of artificial intelligence, machine learning and deep learning using neural networks is extremely broad, and this research article only aims to provide a broad overview in order to demonstrate its application in agriculture and to not discuss these in depth, although many excellent reviews exist for further reading [20–23].

To perform image-to-image translations a similar type of neural network is required, although rather than condensing pixel information into a few datapoints, the shape of the layers more closely resembles that of an hourglass laying on its side (Figure 1a). This hourglass shape allows the network to perform general purpose image-to-image translation and even increase resolution of blurry input images [24]. By pairing this image-to-image transforming network with a second convolutional neural network (Figure 1a), the discriminator, which compares and scores the images produced by the image transforming network and tries to discriminate between fake and ground truth results, a self-training system can be produced. These two-component image translational networks are termed generative adversarial networks (GANs) and have traditionally been used for a variety of image translational tasks including sketch-to-photo, smile-to-frown, and non-bearded-to-bearded [24–26]. More recently, GANs have been applied to medical images to remove noise from low-dose CT, generate tissue structure from blood vessel networks, correct motion artefacts, produce CT images from MR images and synthesise new image data [27–29].

By combining GANs with another machine learning technique, computer vision (CV), any images generated by the GAN can then be analysed to extract data of interest in a fully automated way (Figure 1b). Computer vison is a research field which aims to extract understanding or context from images and can use both traditional mathematical regression techniques as well as deep learning classification networks [30,31]. Application of CV can range from simple inspection of food quality and ripeness by counting the number of pixels within images of fruit and vegetables which fall within certain colour hue ranges [32,33] up to more complex tasks such as identifying road signage or pedestrians to guide automated driving systems [34].

We use both smart techniques (GANs and CV, Figure 1a,b, respectively) to aid processing and analysis of agricultural medical images of sheep. This research aims to first implement a GAN to perform ovine CT processing steps involving global information manipulation such as object and organ removal since within the image are multiple objects (scanning cradle and padding) and organs (testes) of varying size, morphology and orientation. Then, with the processed image containing only key features, attempt to extract phenotypes relevant for breeding programmes using CV techniques in an automated process.

*Sensors* **2021**, *21*, x FOR PEER REVIEW 3 of 18

**Figure 1.** Combined GAN-CV pipeline for phenotype extraction. Neural networks can be trained to perform image-to-image translations such as in (**a**) where a raw ovine CT scan is passed through a generator network, a series of convolution, batch normalisation and ReLu activation function layers, to produce a "fake" image. Skip connections apply regions from the encoded to the encoded images and improve object border definition. By reducing differences between the real and the fake images (green dashes) the autoencoder also learns to better produce fake images independently. A second neural network, the discriminator, then determines if an image is considered real or fake. By pairing the two neural networks to work against each other, an adversarial component emerges, where the generator tries to produces images to fool the discriminator and the discriminator tries to determine if these images are real or not. Phenotype extraction is performed on both real and fake images (**b**) to determine tissue composition and shape before being compared to confirm accuracy.

#### **2. Materials and Methods**
