Motion Prediction and Object Detection for Image-Based Visual Servoing Systems Using Deep Learning
Abstract
:1. Introduction
2. Related Work
2.1. Visual Servoing
2.2. Object Detection
2.3. Deep Learning in Visual Servoing
- Utilizing a DETR for object detection and tracking to enhance the accuracy and speed of visual feedback.
- Implementing PSO to optimize the trajectory of robotic manipulators and developing a unique dataset for training and evaluation.
- Integrating BILSTM and KANs to improve the predictive performance of robot movement.
- Conducting simulations in the ROS environment using Gazebo to verify the effectiveness of the proposed algorithms.
3. Methodology
3.1. Datasets
Dataset for Object Detection
3.2. DETR Model
3.2.1. Object Detection Set Prediction Loss
Algorithm 1: Hungarian algorithm |
|
3.2.2. DETR Architecture
3.3. Trajectory Prediction Model
3.3.1. Particle Swarm Optimization and Polynomial Interpolation Trajectory Planning
- is the velocity of the particle.
- rand() is a uniform random number between 0 and 1.
- is the position of the particle.
- are acceleration coefficients, usually .
- To prevent the velocity from exploding, the velocity is limited to . If the speed exceeds , then .
- A larger value strengthens global search ability and weakens local search ability.
- A smaller value weakens global search ability and strengthens local search ability.
- : maximum number of iterations;
- : initial inertia weight;
- : inertia weight at the maximum number of iterations.
3.3.2. Generating Unique Datasets
3.4. BILSTM-KAN Prediction
3.4.1. BILSTM
3.4.2. Attention Mechanism: Kolmogorov–Arnold Networks
- : input vector of layer l.
- : output vector of layer .
- : activation function connecting the i-th node of layer l to the j-th node of layer .
- : number of nodes in layer l.
- : number of nodes in layer .
3.4.3. BILSTM-KAN Model
- Input Layer: The input layer of the model receives the sequence of spatial pose trajectory coordinates of the robotic arm’s end effector. These coordinates represent the position and orientation of the robotic arm over time, which are crucial for understanding and predicting the arm’s movements.
- BILSTM Layer: The sequence data from the input layer are then fed into a BILSTM layer. The BILSTM layer consists of two LSTM networks: one processes the sequence in the forward direction and the other processes it in the backward direction. This bidirectional processing allows the model to capture dependencies and patterns from both past and future contexts within the sequence, enhancing the learning of temporal dynamics and improving the model’s ability to understand the intricate movements of the robotic arm.
- KAN Layer: The output from the BILSTM layer, which encapsulates the learned features and temporal dependencies, is subsequently fed into a KAN. The KAN layer replaces the traditional fully connected (dense) layer typically used in neural networks. Instead of simple linear transformations, the KAN layer utilizes a combination of MLP and spline functions to transform the input. This approach leverages the representational power of MLPs while incorporating the flexibility and smoothness of spline functions, resulting in a more expressive and interpretable model.
3.5. Visual Servo Simulation Based on ROS and Gazebo
4. Experiments and Results
4.1. ROS and Gazebo Simulation
4.2. Target Detection
- Precision is the ratio of correctly predicted positive observations to the total predicted positives. It is given by
- Recall (also known as sensitivity) is the ratio of correctly predicted positive observations to all observations in the actual class. It is calculated as
4.3. Particle Swarm Optimization (PSO) Trajectory Planning
4.4. BILSTM-KAN Prediction
5. Conclusions
6. Possibilities and Limitations in Industrial Applications
7. Further Work
- Improving Computational Efficiency with KAN: The integration of KANs has proven to be beneficial for enhancing model functionality by facilitating the handling of complex nonlinear relationships. However, this integration has led to increased computational demands. To address this, future investigations could focus on optimizing the structure of KAN layers, perhaps by minimizing the complexity where feasible without significantly compromising performance. Further research could also explore the potential of parallel processing techniques and algorithmic improvements such as network pruning and sparse connectivity. These strategies are anticipated to reduce computational overhead, thus speeding up both the training and inference phases significantly.
- Enhancing Recognition of Small Objects in Complex Backgrounds: Recognizing small objects within complex backgrounds remains a substantial challenge due to the intricate nature of the visual data and limited visibility. Future work could enhance feature extraction capabilities through advanced techniques like scale-invariant feature transforms or employing deep learning architectures specifically tailored for small object detection, such as YOLOv4 or SSD. Additionally, specialized data augmentation methods that simulate small object scenarios in cluttered environments could be developed to train the system more effectively. These methods would prepare the model to perform with higher accuracy and reliability in real-world settings where small object recognition is crucial.
- Transitioning from Simulation to Real-World Applications: To effectively validate the practical utility of the model in real-world scenarios, we plan to conduct a series of progressively advancing experiments to ensure that the model’s performance in actual operations aligns with the results obtained in simulations. This process will be divided into three main stages:
- –
- Preliminary Validation Stage: Closed Testing in a Laboratory EnvironmentIn this stage, we will set up a physical testing platform in a laboratory environment to simulate real-world operating conditions. Specifically, we will use high-precision cameras and industrial robotic arms to simulate various possible scenarios, such as lighting changes, camera noise, and dynamic obstacles.We will first test the model’s basic task execution capabilities under these conditions, such as object recognition, trajectory tracking, etc. Through data collection, we will real-time gather and analyze the model’s performance differences across different scenarios.
- –
- Initial Deployment Stage: Semi-Realistic Scenario Testing in a Controlled EnvironmentAfter laboratory validation, we will deploy the model in a controlled but more realistic environment, such as an industrial production line or automated warehouse. This stage of testing will introduce more complex variables, such as objects of different materials, noise interference in the environment, and unpredictable moving objects.We plan to combine operational data to monitor the model’s performance under different conditions and use these data to fine-tune and enhance the model. Simultaneously, we will collaborate with on-site operators to gather their feedback, assessing the model’s practicality and ease of use.
- –
- Field Validation Stage: Comprehensive Testing in Real-World ApplicationsFinally, we will conduct comprehensive field tests in real-world application scenarios, which may include smart manufacturing, autonomous driving, or unmanned warehousing. At this stage, key performance indicators (e.g., task success rate, response time, system stability, etc.) will become the core criteria for evaluating whether the model has the potential for large-scale application.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Chaumette, F.; Hutchinson, S. Visual servo control. I. Basic approaches. IEEE Robot. Autom. Mag. 2006, 13, 82–90. [Google Scholar] [CrossRef]
- Hutchinson, S.; Hager, G.; Corke, P. A tutorial on visual servo control. IEEE Trans. Robot. Autom. 1996, 12, 651–670. [Google Scholar] [CrossRef]
- Shi, H.; Wu, H.; Xu, C.; Zhu, J.; Hwang, M.; Hwang, K.S. Adaptive Image-Based Visual Servoing Using Reinforcement Learning With Fuzzy State Coding. IEEE Trans. Fuzzy Syst. 2020, 28, 3244–3255. [Google Scholar] [CrossRef]
- Zhu, N.; Xie, W.F.; Shen, H. Position-Based Visual Servoing of a 6-RSS Parallel Robot Using Adaptive Sliding Mode Control. ISA Trans. 2024, 144, 398–408. [Google Scholar] [CrossRef]
- Gubbi, M.R.; Lediju Bell, M.A. Deep Learning-Based Photoacoustic Visual Servoing: Using Outputs from Raw Sensor Data as Inputs to a Robot Controller. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China, 30 May–5 June 2021; pp. 14261–14267. [Google Scholar] [CrossRef]
- Thuilot, B.; Martinet, P.; Cordesses, L.; Gallice, J. Position Based Visual Servoing: Keeping the Object in the Field of Vision. In Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292), Washington, DC, USA, 11–15 May 2002; IEEE: Piscataway, NJ, USA, 2002; Volume 2, pp. 1624–1629. [Google Scholar] [CrossRef]
- Martinet, P.; Gallice, J. Position Based Visual Servoing Using a Non-linear Approach. In Proceedings of the 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289), Kyongju, Republic of Korea, 17–21 October 1999; IEEE: Piscataway, NJ, USA, 1999; Volume 1, pp. 531–536. [Google Scholar] [CrossRef]
- Dong, G.; Zhu, Z. Position-Based Visual Servo Control of Autonomous Robotic Manipulators. Acta Astronaut. 2015, 115, 291–302. [Google Scholar] [CrossRef]
- Park, D.H.; Kwon, J.H.; Ha, I.J. Novel Position-Based Visual Servoing Approach to Robust Global Stability Under Field-of-View Constraint. IEEE Trans. Ind. Electron. 2012, 59, 4735–4752. [Google Scholar] [CrossRef]
- Lippiello, V.; Siciliano, B.; Villani, L. Position-Based Visual Servoing in Industrial Multirobot Cells Using a Hybrid Camera Configuration. IEEE Trans. Robot. 2007, 23, 73–86. [Google Scholar] [CrossRef]
- Parsapour, M.; RayatDoost, S.; Taghirad, H.D. Position Based Sliding Mode Control for Visual Servoing System. In Proceedings of the 2013 First RSI/ISM International Conference on Robotics and Mechatronics (ICRoM), Tehran, Iran, 13–15 February 2013; pp. 337–342. [Google Scholar] [CrossRef]
- Ribeiro, E.G.; Mendes, R.Q.; Terra, M.H.; Grassi, V. Second-Order Position-Based Visual Servoing of a Robot Manipulator. IEEE Robot. Autom. Lett. 2024, 9, 207–214. [Google Scholar] [CrossRef]
- Deng, L. Comparison of Image-Based and Position-Based Robot Visual Servoing Methods and Improvements. Ph.D. Thesis, University of Waterloo, Waterloo, ON, Canada, 2004. [Google Scholar]
- Yang, K.; Bai, C.; She, Z.; Quan, Q. High-Speed Interception Multicopter Control by Image-Based Visual Servoing. arXiv 2024, arXiv:2404.08296. [Google Scholar]
- Albekairi, M.; Mekki, H.; Kaaniche, K.; Yousef, A. An Innovative Collision-Free Image-Based Visual Servoing Method for Mobile Robot Navigation Based on the Path Planning in the Image Plan. Sensors 2023, 23, 9667. [Google Scholar] [CrossRef]
- Zhang, Y.; Yang, Y.; Luo, W. Occlusion-free Image-Based Visual Servoing using Probabilistic Control Barrier Certificates. IFAC-PapersOnLine 2023, 56, 4381–4387. [Google Scholar] [CrossRef]
- Zhu, T.; Mao, J.; Han, L.; Zhang, C. Fuzzy Adaptive Model Predictive Control for Image-Based Visual Servoing of Robot Manipulators with Kinematic Constraints. Int. J. Control Autom. Syst. 2024, 22, 311–322. [Google Scholar] [CrossRef]
- Peng, X.; Li, J.; Li, B.; Wu, J. Constrained Image-Based Visual Servoing of Robot Manipulator with Third-Order Sliding-Mode Observer. Machines 2022, 10, 465. [Google Scholar] [CrossRef]
- Ramani, P.; Varghese, A.; Balachandar, N. Image Based Visual Servoing for Tele-Operated Ground Vehicles. AIP Conf. Proc. 2024, 2802, 110001. [Google Scholar] [CrossRef]
- Tsai, D.; Dansereau, D.G.; Peynot, T.; Corke, P. Image-Based Visual Servoing With Light Field Cameras. IEEE Robot. Autom. Lett. 2017, 2, 912–919. [Google Scholar] [CrossRef]
- McFadyen, A.; Jabeur, M.; Corke, P. Image-Based Visual Servoing With Unknown Point Feature Correspondence. IEEE Robot. Autom. Lett. 2017, 2, 601–607. [Google Scholar] [CrossRef]
- Harish, Y.V.S.; Pandya, H.; Gaud, A.; Terupally, S.; Shankar, S.; Krishna, K.M. DFVS: Deep Flow Guided Scene Agnostic Image Based Visual Servoing. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May–31 August 2020; pp. 9000–9006. [Google Scholar] [CrossRef]
- Machkour, Z.; Ortiz-Arroyo, D.; Durdevic, P. Classical and Deep Learning Based Visual Servoing Systems: A Survey on State of the Art. J. Intell. Robot. Syst. 2022, 104, 11. [Google Scholar] [CrossRef]
- Rekavandi, A.M.; Rashidi, S.; Boussaid, F.; Hoefs, S.; Akbas, E.; Bennamoun, M. Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art. arXiv 2023, arXiv:2309.04902. [Google Scholar]
- Terven, J.; Córdova-Esparza, D.M.; Romero-González, J.A. A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS. Mach. Learn. Knowl. Extr. 2023, 5, 1680–1716. [Google Scholar] [CrossRef]
- Zhang, H.; Li, F.; Liu, S.; Zhang, L.; Su, H.; Zhu, J.; Ni, L.M.; Shum, H.Y. DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. arXiv 2022, arXiv:2203.03605. [Google Scholar]
- El Ahmar, W.; Massoud, Y.; Kolhatkar, D.; AlGhamdi, H.; Alja’afreh, M.; Hammoud, R.; Laganiere, R. Enhanced Thermal-RGB Fusion for Robust Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Vancouver, BC, Canada, 18–22 June 2023; pp. 365–374. [Google Scholar]
- Reis, D.; Kupec, J.; Hong, J.; Daoudi, A. Real-Time Flying Object Detection with YOLOv8. arXiv 2024, arXiv:2305.09972. [Google Scholar]
- Wang, Z.; Li, Y.; Chen, X.; Lim, S.N.; Torralba, A.; Zhao, H.; Wang, S. Detecting Everything in the Open World: Towards Universal Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 17–24 June 2023; pp. 11433–11443. [Google Scholar]
- Katara, P.; Harish, Y.V.S.; Pandya, H.; Gupta, A.; Sanchawala, A.; Kumar, G.; Bhowmick, B.; Krishna, M. DeepMPCVS: Deep Model Predictive Control for Visual Servoing. In Proceedings of the 2020 Conference on Robot Learning, Virtual, 16–18 November 2020; Volume 155, pp. 2006–2015. [Google Scholar]
- Li, J.; Peng, X.; Li, B.; Sreeram, V.; Wu, J.; Chen, Z.; Li, M. Model Predictive Control for Constrained Robot Manipulator Visual Servoing Tuned by Reinforcement Learning. Math. Biosci. Eng. 2023, 20, 10495–10513. [Google Scholar] [CrossRef]
- Fu, G.; Chu, H.; Liu, L.; Fang, L.; Zhu, X. Deep Reinforcement Learning for the Visual Servoing Control of UAVs with FOV Constraint. Drones 2023, 7, 375. [Google Scholar] [CrossRef]
- Lee, A.X.; Levine, S.; Abbeel, P. Learning Visual Servoing with Deep Features and Fitted Q-Iteration. arXiv 2017, arXiv:1703.11000. [Google Scholar]
- Adrian, N.; Do, V.T.; Pham, Q.C. DFBVS: Deep Feature-Based Visual Servo. In Proceedings of the 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), Mexico City, Mexico, 22–26 August 2022; pp. 1783–1789. [Google Scholar] [CrossRef]
- Liu, J.; Li, Y. An Image Based Visual Servo Approach with Deep Learning for Robotic Manipulation. arXiv 2019, arXiv:1909.07727. [Google Scholar]
- He, Y.; Gao, J.; Chen, Y. Deep Learning-Based Pose Prediction for Visual Servoing of Robotic Manipulators Using Image Similarity. Neurocomputing 2022, 491, 343–352. [Google Scholar] [CrossRef]
- Lazo, J.F.; Lai, C.F.; Moccia, S.; Rosa, B.; Catellani, M.; de Mathelin, M.; Ferrigno, G.; Breedveld, P.; Dankelman, J.; De Momi, E. Autonomous Intraluminal Navigation of a Soft Robot using Deep-Learning-Based Visual Servoing. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022; pp. 6952–6959. [Google Scholar] [CrossRef]
- Copot, C.; Shi, L.; Smet, E.; Ionescu, C.; Vanlanduit, S. Comparison of Deep Learning Models in Position Based Visual Servoing. In Proceedings of the 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA), Stuttgart, Germany, 6–9 September 2022; pp. 1–4. [Google Scholar] [CrossRef]
- Abdulhafiz, I.; Nazari, A.A.; Abbasi-Hashemi, T.; Jalali, A.; Zareinia, K.; Saeedi, S.; Janabi-Sharifi, F. Deep Direct Visual Servoing of Tendon-Driven Continuum Robots. In Proceedings of the 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), Mexico City, Mexico, 22–26 August 2022; pp. 1977–1984. [Google Scholar] [CrossRef]
- Jin, Z.; Wu, J.; Liu, A.; Zhang, W.A.; Yu, L. Policy-Based Deep Reinforcement Learning for Visual Servoing Control of Mobile Robots With Visibility Constraints. IEEE Trans. Ind. Electron. 2022, 69, 1898–1908. [Google Scholar] [CrossRef]
- Felton, S.; Fromont, E.; Marchand, E. Deep Metric Learning for Visual Servoing: When Pose and Image Meet in Latent Space. In Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK, 29 May–2 June 2023; pp. 741–747. [Google Scholar] [CrossRef]
- Asayesh, S.; Darani, H.S.; Chen, M.; Mehrandezh, M.; Gupta, K. Toward Scalable Visual Servoing Using Deep Reinforcement Learning and Optimal Control. arXiv 2023, arXiv:2310.01360. [Google Scholar]
- Everingham, M.; Van Gool, L.; Williams, C.K.I.; Winn, J.; Zisserman, A. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. Available online: http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html (accessed on 20 July 2024).
- Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; Zagoruyko, S. End-to-End Object Detection with Transformers. arXiv 2020, arXiv:arXiv.2005.12872. [Google Scholar]
- Wang, W.; Tao, Q.; Cao, Y.; Wang, X.; Zhang, X. Robot Time-Optimal Trajectory Planning Based on Improved Cuckoo Search Algorithm. IEEE Access 2020, 8, 86923–86933. [Google Scholar] [CrossRef]
- Liu, Z.; Wang, Y.; Vaidya, S.; Ruehle, F.; Halverson, J.; Soljačić, M.; Hou, T.Y.; Tegmark, M. KAN: Kolmogorov-Arnold Networks. arXiv 2024, arXiv:2404.19756. [Google Scholar]
- Cao, J.; Pang, Y.; Han, J.; Li, X. Hierarchical Shot Detector. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019. [Google Scholar]
- Zhu, Y.; Zhao, C.; Wang, J.; Zhao, X.; Wu, Y.; Lu, H. CoupleNet: Coupling Global Structure With Local Parts for Object Detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017. [Google Scholar]
- Termritthikun, C.; Jamtsho, Y.; Ieamsaard, J.; Muneesawang, P.; Lee, I. EEEA-Net: An Early Exit Evolutionary Neural Architecture Search. Eng. Appl. Artif. Intell. 2021, 104, 104397. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Computer Vision—ECCV 2016; Springer: Cham, Switzerland, 2016; pp. 21–37. [Google Scholar]
- Dvornik, N.; Shmelkov, K.; Mairal, J.; Schmid, C. BlitzNet: A Real-Time Deep Network for Scene Understanding. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017. [Google Scholar]
Category | Description | Advantages | Disadvantages |
---|---|---|---|
Image-Based Visual Servoing (IBVS) | Uses image features (e.g., points, lines) directly in the control loop to minimize feature error [3]. | Reduces calibration errors; direct feedback from image data. | Sensitive to large camera displacements; may lead to inefficient paths. |
Position-Based Visual Servoing (PBVS) | Uses 3D models to compute camera pose relative to objects, controlling to minimize pose error [4]. | Effective with accurate models; straightforward path planning. | Depends on precise camera calibration and 3D scene knowledge. |
Deep-Learning-Based Visual Servoing | Employs neural networks to predict control actions from images or to enhance feature extraction and pose estimation [5]. | Handles complex environments; improves robustness and generalization. | Requires significant training data and computational power; lacks transparency. |
Authors | Focus of Study | Key Innovations | Practical Applications or Outcomes |
---|---|---|---|
Liu and Li [35] | Robotic Manipulation | Autonomous feature extraction and Jacobian matrix estimation with two-stream CNN | Effective visual servo control for robot manipulators |
He et al. [36] | Pose Prediction for Robotic Manipulators | Pose estimation enhancement using a CNN trained with spherical projection data | Robust performance against occlusion disturbances |
Ribeiro et al. [12] | Robotic Grasping in Dynamic Environments | Training a CNN with the Cornell Grasping Dataset for grasp detection | High accuracy in real-time applications with a Kinova Gen3 manipulator |
Lazo et al. [37] | Intraluminal Navigation in Soft Robots | Movement management using a CNN-trained approach | Demonstrated performance in anatomical phantoms |
Copot et al. [38] | Position-Based Visual Servoing | Comparing deep learning models in visual servoing using a CNN | Effectiveness in precise repositioning tasks, validated in simulations and real environments |
Step | X | Y | Z |
---|---|---|---|
1 | 2.8289 | −40.519 | −7.604 |
5 | 2.8221 | −40.52 | −7.6043 |
9 | 2.7747 | −40.532 | −7.6062 |
13 | 2.6879 | −40.554 | −7.6097 |
17 | 2.3943 | −40.625 | −7.6213 |
21 | 1.9782 | −40.723 | −7.6377 |
25 | 1.3541 | −40.86 | −7.6617 |
29 | 0.47641 | −41.036 | −7.6946 |
33 | −0.70274 | −41.241 | −7.7372 |
37 | −2.2342 | −41.455 | −7.7897 |
41 | −4.1713 | −41.642 | −7.8852 |
Link | d (m) | a (m) | (Rad) | Offset (Rad) |
---|---|---|---|---|
1 | 0.5 | 0.2 | 0 | |
2 | 0 | 0.79 | 0 | |
3 | 0 | 0.14 | 0 | |
4 | 0.78 | 0 | 0 | |
5 | 0 | 0 | 0 | |
6 | 0.104 | 0 | 0 | 0 |
Parameter | camera_rgb_sensor | camera_ir_sensor |
---|---|---|
Link | camera_rgb_link | camera_ir_link |
Type | Depth camera | Depth camera |
FOV | 1.04 | 1.04 |
Image Format | B8G8R8 | L8 |
Resolution | 640 × 480 | 640 × 480 |
Clip Near | 0.01 | 0.01 |
Clip Far | 5 | 5 |
Update Rate | 30 | 30 |
Point Cloud Cutoff | 0.1 | 0.1 |
Name | Model | Epochs | Mini Batch Size | Learning Rate | Image Size | MAP (%) |
---|---|---|---|---|---|---|
Hierarchical Shot Detector [47] | VGG16 | 250 | 32 | 0.004, decreases by a factor of 10 at 150 and 200 epochs | 83.00 | |
Coupling Global Structure with Local Parts [48] | CoupleNet | - | 2 | for 80k, then for 30k iterations | resized from 480 to 864 | 82.70 |
Early Exit Evolutionary Neural Architecture [49] | EEEA-Net-C2 | 200 | 32 | initial 0.01, then for 40k, for 10k | 81.80 | |
Single Shot MultiBox Detector [50] | SSD512 | - | 32 | 0.001 for 40k, then 10k iterations with and | 81.60 | |
BlitzNet [51] | BlitzNet512 | - | 32 | initial learning rate is and decreases twice during training by a factor 10. | 81.50 | |
DETR | DETR | 300 | 8 | initial , cosine decay | 84.47 |
Hyperparameter | Value |
---|---|
Control Period | 0.05 |
Number of Iterations | 200 |
Number of Particles | 20 |
Dimension of the Problem | 3 |
Inertia Weight | 0.7 |
Cognitive Coefficient | 4 |
Social Coefficient | 4 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hao, Z.; Zhang, D.; Honarvar Shakibaei Asli, B. Motion Prediction and Object Detection for Image-Based Visual Servoing Systems Using Deep Learning. Electronics 2024, 13, 3487. https://doi.org/10.3390/electronics13173487
Hao Z, Zhang D, Honarvar Shakibaei Asli B. Motion Prediction and Object Detection for Image-Based Visual Servoing Systems Using Deep Learning. Electronics. 2024; 13(17):3487. https://doi.org/10.3390/electronics13173487
Chicago/Turabian StyleHao, Zhongwen, Deli Zhang, and Barmak Honarvar Shakibaei Asli. 2024. "Motion Prediction and Object Detection for Image-Based Visual Servoing Systems Using Deep Learning" Electronics 13, no. 17: 3487. https://doi.org/10.3390/electronics13173487