*4.2. Experiment Results*

#### 4.2.1. Pallet Monitoring Module

In order to test the monitoring effect of pallets from multiple cameras, we placed pallets at different locations in the 8-way camera field of view for actual testing. Considering that pallets are likely to be stacked and loaded with goods at customer sites, we placed 2 and 3 layers of pallets in a large number of pallets and placed goods at the top edge of the pallets. We used the side aperture feature of the pallets, which is not easily concealed, as the identification mark in order to prevent the influence of the goods on the pallets on the recognition results. The placement of the pallets under each camera is shown in Figure 7.

**Figure 7.** Pallet placement under each camera. Different number of pallets with different number of layers were placed under different cameras and tested repeatedly.

The figure shows how the pallet monitoring system can accurately present the results of pallet recognition for multiple scenarios. Our algorithm places a red box in the side hole position of the recognized pallet to show that a pallet was identified. To see the recognition results more clearly, we use the camera 8-th with the highest number of pallets for analysis.

Before adding the small target detection and CBAM described in 3-A, the recognition effect of camera 8-th is shown in Figure 8. It is obvious that some pallets located further away from the camera cannot be detected, which is typically caused by unreliable recognition brought on by low pixels.

**Figure 8.** The recognition effect of camera 8-th. The majority of the pallets in the field of view are identified, however certain specific pallets are still not.

We have used small target detection and CBAM to solve such problems, and the recognition effect after addition is shown in Figure 9. It can be clearly seen that the unrecognized pallets in Figure 8 have been recognized in Figure 9, and all unobstructed pallets in the field of view are recognized.

**Figure 9.** Recognition effect after using small target detection and CBAM. All pallets in the field of view, including multi-layer pallets and cargo pallets, are recognized.

To better reflect the advantages of our algorithm, we list the comparison of existing algorithms with our algorithm in Table 1.


**Table 1.** Comparison of pallet monitoring algorithms.

As can be seen from the table, the implementation cost of the photoelectric sensor method is high; the traditional image method is susceptible to interference; and the deep learning method based on Yolov3 does not have a high recognition rate for small targets. Although our method, with low cost, still has a high recognition rate for small targets.

In order to test the reliability of the system operation, we designed the experiment for continuous 7 × 24 h non-stop testing. We adjust the position of multi-layer pallets and goods within the scene before the start of each day's experiment to ensure that the conditions are different each time. Following the change, we will count the unobstructed pallets and enter that number as a parameter in the system. The system will calculate the recognition rate by comparing the pallets it has identified with the input parameter. Table 2 shows the number of pallets and the recognition rate during the 7-day experiments.


**Table 2.** Pallet monitoring reliability experiments.

As can be seen in Table 2, the recognition rate is less than 100%, indicating that there are still cases where individual pallets are not recognized. The recognition rate exceeds 99.5% for about 70 pallets tested continuously for 7 days, including single-layer, multi-layer, with or without goods. For the situation where a large number of pallets appear under one camera, recognition can still be performed stably as long as the pallets are not blocked from each other before (i.e., a reasonable distance between pallets is set). The eight cameras are able to output data streams together for simultaneous recognition of pallets in the area.

#### 4.2.2. Pallet Positioning Module and High Precision Control Module

In the pallet position recognition module, we use RGB-D camera to collect data from several different positions and angles of the pallet for calculation, and provide the results to the intelligent forklift to control the forklift for precise pallet insertion. Since we expect to obtain the overall accuracy of pallet insertion by the forklift, we do not distinguish the positional accuracy from the control accuracy; instead, we judge the ultimate total insertion accuracy.

We collect a lot of data in order to ensure that the findings of our trials covered the majority of usage cases. Given that the relationship between forklifts and pallets is typically not fixed before pallets are inserted, we set up a variety of starting positions: the distances between the pallet hole plane and camera plane are 1 m, 2 m, and 3 m; the horizontal distances are 0 m, −0.2 m, and +0.2 m; and the relative angles between the camera and pallet are 0deg, +15 deg, and −15 deg. Additionally, we established two separate heights, with the pallet placed on the ground being recorded as 0, and the pallet placed on a shelf that is 40 cm high being recorded as 0.4 m. This is because a single pallet may be placed on either the ground or a shelf. We carried out pallet docking approximately 10 times for each scenario, comparing the inaccuracy of each result with the initial result. In other words, we completed a total of 1000 dockings, of which 550 were for the 1-layer pallets, 180 were for the 2-layer pallets, and 260 were for the 3-layer pallets.

We have placed the pallets in different positions and layers and recorded the final error. We recorded the error value of each result compared with the first result and drew it in Figure 10, where the x-axis represents the number of docking times and the y-axis represents the result error value.

**Figure 10.** Results of pallet insertion experiment. 1-layer pallets test 550 times, 2-layer pallets test 180 times, 3-layer pallets test 260 times, and the maximum error of 6 mm.

Figure 10 shows that the accuracy in both the x-direction and the y-direction can be controlled to about 5 mm. However, in some circumstances, particularly when dealing with three-layer pallets, the error will increase to 6 mm because multi-layer pallets produce more interference than single-layer pallets.

Overall, our pallet docking accuracy can be controlled within ±6 mm and the coverage range is from 1–3 m and from −15° to +15°. When compared to the measures in the literature [32], the coverage range is roughly the same, but it does not test for severe pallet angle deflection. More importantly, the pallet accuracy in the literature [32] is only ±3 cm, which is much lower than our ±6 mm accuracy. Similar coverage and coverage angle experiments to ours were carried out in the literature [33], but their maximum error of recognition was 10.5 mm after only 135 experiments, which was 75% higher than our maximum error of 6 mm. Its mean error and standard deviation are also significantly higher than ours. Meanwhile, our number of experiments was 7.4 times higher than his, and we measured the final error after pallet docking, which is the accumulation of recognition error and control error values, indicating that our identification error values are lower. On the other hand, the literature [33] used two sensors to achieve the results expressed in the article, while we obtained better results by using only one RGB-D camera. The experiment results are shown in Table 3.


**Table 3.** The effect comparison of three algorithms.

The data in the table shows that the maximum error of the existing algorithm for pallet position recognition can be 1–3 cm, which is a risk that the fork tines cannot enter the pallet hole for the high-precision pallet insertion action of the forklift. Our module is able to reduce the maximum error to 6 mm, which fully meets the needs of practical use. At the same time, our system requires only one RGB-D camera to collect data without the assistance of other sensors, which is much less costly.This demonstrates that our approach outperforms the majority of systems now used in industrial settings in terms of accuracy, resilience, and cost.

According to the above experimental results, the recognition rate of our pallet monitoring system has been significantly improved after the addition of small target detection, and the recognition rate reached 99.5% under 7 days of continuous experiments; our position recognition and control algorithms work together to control the forklift interpolation accuracy within 6 mm in 1000 experiments, which fully meets the requirements of accuracy in practical use and outperforms existing algorithms in both coverage and accuracy.
