Author Contributions
Conceptualization, J.D., D.B. and G.N.D.; Methodology, G.O., S.M.A.T. and G.N.D.; Software, G.O. and S.M.A.T.; Validation, G.O. and S.M.A.T.; Formal analysis, G.O. and S.M.A.T.; Investigation, G.O. and S.M.A.T.; Resources, G.O. and S.M.A.T.; Data curation, G.O. and S.M.A.T.; Writing—original draft, G.O. and S.M.A.T.; Writing—review & editing, G.O., S.M.A.T. and G.N.D.; Visualization, G.O. and S.M.A.T.; Supervision, J.D., D.B. and G.N.D.; Project administration, J.D., D.B. and G.N.D. All authors have read and agreed to the published version of this manuscript.
Figure 1.
(a) A schematic representation of the scanning system. (b) Real-life figure of the camera frame and the system components.
Figure 1.
(a) A schematic representation of the scanning system. (b) Real-life figure of the camera frame and the system components.
Figure 2.
Overview of the software pipeline: The pipeline begins with the data acquisition of RGBD data, which undergo a segmentation and filtering step to eliminate the background pixels and noise in both depth and RGB space. The filtered data are subsequently backprojected into 3D space and then stitched to form a unified 3D model. A mesh is then constructed over the 3D point cloud. Finally, we measure our traits of interest, volume, and surface area.
Figure 2.
Overview of the software pipeline: The pipeline begins with the data acquisition of RGBD data, which undergo a segmentation and filtering step to eliminate the background pixels and noise in both depth and RGB space. The filtered data are subsequently backprojected into 3D space and then stitched to form a unified 3D model. A mesh is then constructed over the 3D point cloud. Finally, we measure our traits of interest, volume, and surface area.
Figure 3.
Schematic layout of Server–Client: In this configuration, the Client sends a capture request to 10 Server programs. Each Server program performs the image acquisition request from the Client and the captured data are transmitted to a storage device.
Figure 3.
Schematic layout of Server–Client: In this configuration, the Client sends a capture request to 10 Server programs. Each Server program performs the image acquisition request from the Client and the captured data are transmitted to a storage device.
Figure 4.
Mask R-CNN Architecture [
9]: Mask R-CNN builds upon two existing Faster R-CNN heads as detailed in [
10,
11]. The left and right panels illustrate the heads for the ResNet C4 and FPN backbones, respectively, with an added mask branch. Spatial resolution and channels are indicated by the numbers, while arrows represent conv, deconv, or FC layers, inferred from the context (conv layers maintain spatial dimensions, whereas deconv layers increase them). All conv layers are 3 × 3, except for the output conv which is 1 × 1. Deconv layers are 2 × 2 with a stride of 2, and ReLU [
12] is used in hidden layers. On the left, ‘res5’ refers to the fifth stage of ResNet, which has been modified so that the first conv layer operates on a 7 × 7 RoI with a stride of 1 (instead of 14 × 14 with a stride of 2 as in [
10]). On the right, ‘×4’ indicates a stack of four consecutive conv layers.
Figure 4.
Mask R-CNN Architecture [
9]: Mask R-CNN builds upon two existing Faster R-CNN heads as detailed in [
10,
11]. The left and right panels illustrate the heads for the ResNet C4 and FPN backbones, respectively, with an added mask branch. Spatial resolution and channels are indicated by the numbers, while arrows represent conv, deconv, or FC layers, inferred from the context (conv layers maintain spatial dimensions, whereas deconv layers increase them). All conv layers are 3 × 3, except for the output conv which is 1 × 1. Deconv layers are 2 × 2 with a stride of 2, and ReLU [
12] is used in hidden layers. On the left, ‘res5’ refers to the fifth stage of ResNet, which has been modified so that the first conv layer operates on a 7 × 7 RoI with a stride of 1 (instead of 14 × 14 with a stride of 2 as in [
10]). On the right, ‘×4’ indicates a stack of four consecutive conv layers.
Figure 5.
Multi-view point cloud registration: (a) Given N = 6 point clouds, we perform a simple pairwise registration of point cloud fragments of the scanned cattle. (b) We use the Colored ICP algorithm to solve for the coordinate transformation from camera coordinate frame j to camera coordinate frame i (denoted as ). Each view is aligned into the coordinate frame of its adjacent camera. We fix the coordinate frame of Camera 1 () as the world coordinate frame and then align all views with respect to coordinate frame 1. (c) This results in a well-aligned point cloud of the scanned cattle.
Figure 5.
Multi-view point cloud registration: (a) Given N = 6 point clouds, we perform a simple pairwise registration of point cloud fragments of the scanned cattle. (b) We use the Colored ICP algorithm to solve for the coordinate transformation from camera coordinate frame j to camera coordinate frame i (denoted as ). Each view is aligned into the coordinate frame of its adjacent camera. We fix the coordinate frame of Camera 1 () as the world coordinate frame and then align all views with respect to coordinate frame 1. (c) This results in a well-aligned point cloud of the scanned cattle.
Figure 6.
Comparison of 3D point cloud capture quality with and without synchronization using a large box with known dimensions. The left image displays the results without synchronization (0 μs), capturing a total of 17,098 points. The right image shows the same box captured with synchronization (160 μs) with all other settings the same, resulting in a total of 38,631 points, illustrating the significant improvement in data acquisition quality. (a) Large box, 0 s delay, n = 17,098. (b) Large box, 160 μs delay, n = 38,631.
Figure 6.
Comparison of 3D point cloud capture quality with and without synchronization using a large box with known dimensions. The left image displays the results without synchronization (0 μs), capturing a total of 17,098 points. The right image shows the same box captured with synchronization (160 μs) with all other settings the same, resulting in a total of 38,631 points, illustrating the significant improvement in data acquisition quality. (a) Large box, 0 s delay, n = 17,098. (b) Large box, 160 μs delay, n = 38,631.
Figure 7.
Results of scanning a cylindrical object in multiple orientations, highlighting the scanner’s accuracy across diverse poses. The horizontal axis displays the predicted volumes and surface areas obtained in each test. Given that the same object was used throughout, the ground truth volume and surface area remain constant. This plot demonstrates the scanner’s precision, as evidenced by the close alignment of the predicted values with the consistent ground truths, illustrating the system’s reliability in varying orientations. (a) Surface area calculation results. (b) Volume calculation results.
Figure 7.
Results of scanning a cylindrical object in multiple orientations, highlighting the scanner’s accuracy across diverse poses. The horizontal axis displays the predicted volumes and surface areas obtained in each test. Given that the same object was used throughout, the ground truth volume and surface area remain constant. This plot demonstrates the scanner’s precision, as evidenced by the close alignment of the predicted values with the consistent ground truths, illustrating the system’s reliability in varying orientations. (a) Surface area calculation results. (b) Volume calculation results.
Figure 8.
Regression analysis of predicted versus known surface area and volume for multiple static objects. The plots displays the correlation between the scanner’s predicted values and the actual measurements for a cylinder, small box, medium box, and large box, all placed in the same pose across 10 consecutive scans. The high values of 0.997 for surface area and 0.999 for volume demonstrate the scanner’s accuracy and consistency in various object dimensions and shapes under controlled conditions. (a) Surface area calculation results. (b) Volume calculation results.
Figure 8.
Regression analysis of predicted versus known surface area and volume for multiple static objects. The plots displays the correlation between the scanner’s predicted values and the actual measurements for a cylinder, small box, medium box, and large box, all placed in the same pose across 10 consecutive scans. The high values of 0.997 for surface area and 0.999 for volume demonstrate the scanner’s accuracy and consistency in various object dimensions and shapes under controlled conditions. (a) Surface area calculation results. (b) Volume calculation results.
Figure 9.
Performance of the scanner under direct sunlight, using a standard box to simulate outdoor livestock scanning conditions. The graphs show the mean and standard deviation of volume and surface area measurements across 10 consecutive scans. The results here illustrate the slight impact of sunlight on the scanner’s infrared sensors, affecting measurement accuracy. (a) Surface area calculation results from data collected in sunlight. (b) Volume calculation results from data collected in sunlight.
Figure 9.
Performance of the scanner under direct sunlight, using a standard box to simulate outdoor livestock scanning conditions. The graphs show the mean and standard deviation of volume and surface area measurements across 10 consecutive scans. The results here illustrate the slight impact of sunlight on the scanner’s infrared sensors, affecting measurement accuracy. (a) Surface area calculation results from data collected in sunlight. (b) Volume calculation results from data collected in sunlight.
Figure 10.
Segmentation of cattle using combined RGB and depth models via Mask R-CNN: The figure shows an RGBD image of cattle segmented using both RGB and depth data. Results from each model are integrated using a voting arbitrator, resulting in a well-defined segmentation in both modalities.
Figure 10.
Segmentation of cattle using combined RGB and depth models via Mask R-CNN: The figure shows an RGBD image of cattle segmented using both RGB and depth data. Results from each model are integrated using a voting arbitrator, resulting in a well-defined segmentation in both modalities.
Figure 11.
Poisson reconstructed meshes of cattle from which we compute the surface area and volume estimates.
Figure 11.
Poisson reconstructed meshes of cattle from which we compute the surface area and volume estimates.
Table 1.
Measurements of the number of points generated in the synchronized and unsynchronized mode of the ToF sensor.
Table 1.
Measurements of the number of points generated in the synchronized and unsynchronized mode of the ToF sensor.
Sensor ID | Number of Points Synchronized (160 μs) | Number of Points Unsynchronized (0 μs) |
---|
Camera 1 | 33,607 | 36,062 |
Camera 2 | 16,127 | 15,235 |
Camera 3 | 15,333 | 13,937 |
Camera 4 | 24,743 | 22,969 |
Camera 5 | 28,597 | 25,573 |
Camera 6 | 32,044 | 27,303 |
Camera 7 | 42,803 | 29,432 |
Camera 8 | 37,469 | 35,046 |
Camera 9 | 37,583 | 35,149 |
Camera 10 | 38,631 | 17,098 |
Table 2.
The statistical results of varying the distance of the object to the ToF sensor.
Table 2.
The statistical results of varying the distance of the object to the ToF sensor.
Distance to Sensor | Surface Area NFOV (m2) | Surface Area WFOV (m2) | Number of Points NFOV | Number of Points WFOV |
---|
2ft | 0.290 | 0.294 | 2815 | 3058 |
3ft | 0.349 | 0.343 | 3015 | 3253 |
4ft | 0.278 | 0.280 | 2869 | 3135 |
5ft | 0.315 | 0.335 | 3202 | 3442 |
6ft | 0.277 | 0.283 | 2882 | 3153 |
7ft | 0.251 | 0.254 | 2731 | 3010 |
Table 3.
The statistical results (the average IOU, the false positive rate (FR), and the false negative rate (FN)) of using different voting arbitration for the segmentation models. The bold font specifies the best scores among the others.
Table 3.
The statistical results (the average IOU, the false positive rate (FR), and the false negative rate (FN)) of using different voting arbitration for the segmentation models. The bold font specifies the best scores among the others.
Voting Arbitration | Avg. IOU | FP Rate (%) | FN Rate (%) |
---|
RGB only | 0.9653 | 1.4480 | 3.7249 |
Depth only | 0.9495 | 2.6401 | 4.9985 |
1-Vote (OR) | 0.9629 | 3.2592 | 2.2631 |
2-Vote (AND) | 0.9518 | 0.7655 | 6.4603 |
Table 4.
Comparison between the measured surface area using the proposed 3D cattle scanner and the hand measurements from the hide of slaughtered animals. Each of the animals (which is determined by its cattle ID) was scanned 5 times and the average value was calculated over the scans.
Table 4.
Comparison between the measured surface area using the proposed 3D cattle scanner and the hand measurements from the hide of slaughtered animals. Each of the animals (which is determined by its cattle ID) was scanned 5 times and the average value was calculated over the scans.
Cattle ID | Manual-Measured Surface Area (m2) | Average Estimated Surface Area (m2) | Surface Area Std (m2) | Average Error (%) | Average Processing Time (s) |
---|
1 | 5.316 | 5.52270 | 0.09810 | 3.887 | 10.64 |
5 | 5.088 | 5.03896 | 0.09579 | 1.482 | 11.02 |
7 | 5.189 | 5.63482 | 0.08180 | 8.591 | 11.04 |
8 | 5.202 | 5.39062 | 0.06005 | 3.625 | 10.89 |
13 | 5.660 | 5.55332 | 0.12395 | 2.229 | 11.11 |
14 | 5.406 | 5.50881 | 0.04657 | 1.901 | 11.63 |
15 | 5.024 | 5.48532 | 0.05472 | 9.182 | 10.90 |
18 | 5.329 | 5.42638 | 0.09839 | 1.827 | 11.22 |
21 | 5.278 | 5.54710 | 0.06522 | 5.098 | 10.43 |
22 | 5.571 | 5.34380 | 0.15375 | 4.078 | 10.74 |
Average | - | - | 0.08783 | 4.1906 | 10.96 |