Depth Image Selection Based on Posture for Calf Body Weight Estimation ^†

Abstract

We are developing a system to estimate body weight using calf depth images taken in a loose barn. For this purpose, depth images should be taken from the side, without calves overlapping and without their backs bent. However, most of the depth images that are taken successively and automatically do not satisfy these conditions. Therefore, we need to select only the depth images that match these conditions, as to take many images as possible. The existing method assumes that a calf standing sideways and upright in front of cameras is in a suitable pose. However, since such cases rarely occur, not many images were selected. This paper proposes a new depth image-selection method, focusing on whether a calf is sideways, and the back is not bent, regardless of whether the calf is still or walking. First, depth images including only a single calf are extracted. The calf was identified using radio frequency identification (RFID) when its depth image was taken. Then, the calf area was extracted by background subtraction and contour detection with a depth image. Finally, to judge the usable depth images, we detected and evaluated the calf’s posture, such as the angle of the calf to the camera and the slope of the dorsal line. We used the mean absolute percentage error (MAPE) to assess the efficiency of our method. As two times the number of depth images were extracted, our method achieved an MAPE of 12.45%, while the existing method achieved an MAPE of 13.87%. From this result, we have confirmed that our method makes body weight estimation more accurate.

1. Introduction

There is a high demand to select the breeding stock with a high reproductive ability in cattle breeding farms. The reproductive ability can be measured as the maternal ability as well as the calving interval. The weight changes in calves are often used as an indicator of maternal ability [1]. However, manually measuring the calf’s weight burdens farmers and stresses calves. Hence, several methods using computer vision were studied to automate this process [2]. We are developing a system to estimate body weight using calf depth images taken in a loose barn, where the calves can move without stress. Depth images appropriate for body weight estimations are ones taken from the side, without calves overlapping and without their backs bent. In [3], a method to select such depth images from all those that were taken has been proposed, under the consideration that standing sideways and still upright in front of a camera is a suitable pose. However, such cases rarely occurred, since the calves were walking around freely. As a result, few images were selected.

Therefore, this paper proposes a method to select appropriate depth images regardless of whether calves are still or walking, as some depth images taken when a calf was walking across in front of the camera can still be useful. This method does not utilize any constraint condition to judge whether the calf is still and upright but focuses on whether the calf is sideways and the back is not bent. When detecting the calf’s posture, such as the angle of the calf to the camera and the slope of the dorsal line, our method is more accurate than the previous method, as it corrects the calf’s contour extracted from a depth image and considers the local depth of body parts.

2. Methodology

2.1. Overview of Calf Body Weight Estimation System

Figure S1 shows the process flow of our automatic calf body weight estimation system in a loose barn. First, depth images including only a single calf are selected according to an individual identification, using radio frequency identification (RFID). Next, the calf’s area is cut out from the depth image. If it is judged that the calf’s posture is appropriate for body weight estimation, its body part is modeled, because chest girth and waist girth have a high correlation with body weight, and its volume is calculated. Finally, the body weight is estimated based on the volume [4,5].

2.2. Individual Identification by Using RFID

The RFID system can read multiple electronic tags at once. The tag reader that we used is RWLU1002, manufactured by MASPRO DENCO Corp., using a specific low-power 920 MHz radio with a maximum transmission power of 24 dBm. The reader antenna is RAF4031, also manufactured by MASPRO DENCO Corp., with a narrow half-value angle of 22 degrees. Two RFID tags hang down each side from a belt around a calf’s neck, which means that the calf can be identified from its right and left sides. In addition, the RFID reader antenna and the camera are aligned in the same direction. Therefore, by comparing the shooting time with the identification time, how many calves are taken in the image and what each calf is can be known, which enables the depth image including only a single calf to be selected. However, this selection does not always succeed, especially when other calves’ RFID tags are hidden.

2.3. Selecting Depth Images Based on Calf’s Posture

The calf’s body part is modeled using a three-dimensional successive cylindrical model vertical to the direction of the camera [4,5]. When the axis of the body is inclined, i.e., the angle of the calf to the camera is not vertical or the calf’s dorsal line is not horizontal, the body part cannot accurately be modeled (Figures S2 and S3). Therefore, these images need to be excluded. For this posture judgment, the calf’s area on the depth image is extracted using the background subtraction method [6].

The angle between the calf and the camera is estimated based on the average depth values in each part: head, shoulder top, and rump (Figure 1a). Since calves move their heads freely, even if they are not inclined from rump to head, their body parts may be bent, which leads to a less accurate weight estimation. For this reason, not only the entire body, but also each part of the body, is checked to ensure it is straight and sideways. It is critical to accurately record the angle of the calf’s axis using the depth values near the backbone, which is less affected by bulges on the sides of the body. The slope of the calf’s dorsal line is estimated to be that of the line from the root of the neck to the rump (Figure 1b). The issue in this process is that depth data near the contour are likely to lose the depth image of a moving calf, so that the slope of the dorsal line may not be accurately detected. Therefore, the dorsal contour is complementarily smoothed before dorsal slope detection.

3. Experiment

3.1. Data Set

To assess the effectiveness of the proposed method, we conducted experiments to compare it with an existing image-selection method [3]. In the experiment, we targeted calves, from newborns to calves weighing under 100 kg, at the Food Resources Education and Research Center of Kobe University. The depth camera we used is RealSense Depth Camera D415 manufactured by Intel. Using RFID, 65,000 images were judged to include only one calf among those taken from 6 December 2020 to 25 January 2021. Note that 463 images included multiple calves rather than a single calf. Among these, 45,000 images taken from 6 December 2020 to 7 January 2021, and 20,000 images taken from 8 January 2020 to 25 January 2021, were used for training and testing, respectively, for body weight estimation [3]. Two image-selection methods, i.e., the existing method [3] and the proposed method, were applied for those images. In the training phase, first, body volumes were estimated using the images chosen by each image-selection method. Then, a linear regression equation between the calculated volumes and the manually measured body weights, and a coefficient of determination, were determined. In the testing phase, the body volumes were estimated using the images chosen by each image-selection method. Then, the body weights were calculated using the regression equation obtained in the training phase. Finally, we calculated mean absolute percentage error (MAPE) between estimated body weight $W_i^{\mathrm{estimated}}$ and actual body weight $W_i^{\mathrm{actual}}$:

$$MAPE=\frac{\left|W_i^{\mathrm{actual}}-W_i^{\mathrm{estimated}}\right|}{W_i^{\mathrm{actual}}}\times 100.$$

(1)

Note that only circle fitting was used to model the calf’s body part to calculate the volume [4,5]. Moreover, when obtaining multiple images of one calf on the same day, we used the average of the volumes, because this improves the accuracy of body weight estimation.

3.2. Result

Table 1. The experimental results for each of the two methods.

Method	Number of Images/Calves (Training Data)	Number of Images/Calves (Test Data)	Coefficient of Determination	MAPE (%)
Existing method [3]	375/60	69/19	0.6303	13.87
Proposed method	650/78	226/27	0.6548	12.45

shows the experimental results for each of the two methods: the number of images selected, the total number of calf-days, the coefficient of determination, and MAPE.

4. Discussion

Thanks to the posture judgment, which focuses on the angle to the camera and the slope of the dorsal line, we could acquire almost twice as many depth images by using the proposed method as by using the previous method. On the other hand, the existing method had a larger MAPE, even though depth images of an upright calf accounted for a larger percentage of the selected images. From these results, we consider it crucial to acquire more depth images, instead of selecting only high-quality ones. Thus, we have confirmed that our method makes body weight estimation more accurate. The proposed method and the existing method have selected four depth images and two depth images with multiple calves overlapping, respectively (Figure S4). Therefore, such images should be eliminated using outlier detection.

5. Conclusions

This paper proposed an image selection method to choose appropriate depth images, regardless of whether calves are still or walking, for body weight estimation. We have confirmed that our method acquires more depth images from the experimental results and makes the body weight estimation more accurate. In future work, we should establish a method to eliminate inappropriate depth images. It also needs to be tested on other farms to verify its versatility. For this to operate on a farm, we plan to expand the research on calf body weight estimation.