Hardness Classification Using Cost-Effective Off-the-Shelf Tactile Sensors Inspired by Mechanoreceptors
Abstract
:1. Introduction
2. Background
2.1. Mechanoreceptors
2.2. Tactile Sensors
2.3. Material Classification
2.4. Shore Hardness Scale
2.5. Summary
3. Approach
3.1. Identification of Mechanoreceptor-Inspired Tactile Sensors
3.2. Hardness Classification Using Mechanoreceptor-Inspired COTS Sensors
3.3. Machine Learning Approach for Hardness Classification
4. Design of Experiment
4.1. Object Preparation
4.2. Hardware
4.3. Software
4.4. Gripper System
4.5. Control System
4.6. Pressure System
4.7. Data Collection
5. Results Based on Machine Learning
5.1. Result on Hardness Classification Outcome Based on Two Classes (H,S)
5.2. Hardness Classification Outcome Based on Three Classes (H,S,F)
5.3. Result from Best Algorithm and Sensors Configuration—Multiclass Output
6. Conclusions and Future Scope
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Jin, J.; Wang, S.; Zhang, Z.; Mei, D.; Wang, Y. Progress on flexible tactile sensors in robotic applications on objects properties recognition, manipulation and human-machine interactions. Soft Sci. 2023, 3, 8. [Google Scholar] [CrossRef]
- Eguíluz, A.G.; Rañó, I.; Coleman, S.A.; McGinnity, T.M. Multimodal Material identification through recursive tactile sensing. Robot. Auton. Syst. 2018, 106, 130–139. [Google Scholar] [CrossRef]
- Yi, Z.; Zhang, Y.; Peters, J. Bioinspired tactile sensor for surface roughness discrimination. Sens. Actuators A Phys. 2017, 255, 46–53. [Google Scholar] [CrossRef]
- Li, G.; Liu, S.; Wang, L.; Zhu, R. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Sci. Robot. 2020, 5, 46–53. [Google Scholar] [CrossRef]
- Li, F.; Wang, R.; Song, C.; Zhao, M.; Ren, H.; Wang, S.; Liang, K.; Li, D.; Ma, X.; Zhu, B.; et al. A Skin-Inspired Artificial Mechanoreceptor for Tactile Enhancement and Integration. ACS Nano 2021, 15, 16422–16431. [Google Scholar] [CrossRef] [PubMed]
- Iheanacho, F.; Vellipuram, A.R. Physiology, Mechanoreceptors; StatPearls Publishing: Tampa, FL, USA, 2023. [Google Scholar]
- Dahiya, R.; Oddo, C.; Mazzoni, A.; Jörntell, H. Biomimetic tactile sensing. In Biomimetic Technologies; Elsevier: Amsterdam, The Netherlands, 2015; pp. 69–91. [Google Scholar] [CrossRef]
- Amin, Y.; Gianoglio, C.; Valle, M. Embedded real-time objects’ hardness classification for robotic grippers. Future Gener. Comput. Syst. 2023, 148, 211–224. [Google Scholar] [CrossRef]
- Song, Y.; Lv, S.; Wang, F.; Li, M. Hardness-and-Type Recognition of Different Objects Based on a Novel Porous Graphene Flexible Tactile Sensor Array. Micromachines 2023, 14, 217. [Google Scholar] [CrossRef]
- Qian, X.; Li, E.; Zhang, J.; Zhao, S.-N.; Wu, Q.-E.; Zhang, H.; Wang, W.; Wu, Y. Hardness Recognition of Robotic Forearm Based on Semi-supervised Generative Adversarial Networks. Front. Neurorobot. 2019, 13, 73. [Google Scholar] [CrossRef] [PubMed]
- Jamali, N.; Sammut, C. Material classification by tactile sensing using surface textures. In Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA, 3–7 May 2010; IEEE: New York, NY, USA, 2010; pp. 2336–2341. [Google Scholar]
- Konstantinova, J.; Cotugno, G.; Stilli, A.; Noh, Y.; Althoefer, K. Object classification using hybrid fiber optical force/proximity sensor. In 2017 IEEE SENSORS; IEEE: New York, NY, USA, 2017; pp. 1–3. [Google Scholar] [CrossRef]
- Dai, K.; Wang, X.; Rojas, A.M.; Harber, E.; Tian, Y.; Paiva, N.; Gnehm, J.; Schindewolf, E.; Choset, H.; Webster-Wood, V.A.; et al. Design of a Biomimetic Tactile Sensor for Material Classification. arXiv 2022, arXiv:2203.15941. [Google Scholar]
- Madrigal, D.; Torres, G.; Ramos, F.; Vega, L. Cutaneous mechanoreceptor simulator. In Proceedings of the 2012 IEEE 3rd International Conference on Cognitive Infocommunications (CogInfoCom), Kosice, Slovakia, 2–5 December 2012; IEEE: New York, NY, USA, 2012; pp. 781–786. [Google Scholar] [CrossRef]
- Najarian, S.; Dargahi, J.; Mehrizi, A.A. Artificial Tactile Sensing in Biomedical Engineering; McGraw-Hill Education: New York, NY, USA, 2009. [Google Scholar]
- Chun, S.; Kim, J.-S.; Yoo, Y.; Choi, Y.; Jung, S.J.; Jang, D.; Lee, G.; Song, K.-I.; Nam, K.S.; Youn, I.; et al. An artificial neural tactile sensing system. Nat. Electron. 2021, 4, 429–438. [Google Scholar] [CrossRef]
- Pattnaik, D.; Sharma, Y.; Saveliev, S.; Borisov, P.; Akther, A.; Balanov, A.; Ferreira, P. Stress-induced Artificial neuron spiking in Diffusive memristors. arXiv 2023, arXiv:2306.12853. [Google Scholar]
- Kalita, H.; Krishnaprasad, A.; Choudhary, N.; Das, S.; Dev, D.; Ding, Y.; Tetard, L.; Chung, H.-S.; Jung, Y.; Roy, T. Artificial Neuron using Vertical MoS2/Graphene Threshold Switching Memristors. Sci. Rep. 2019, 9, 53. [Google Scholar] [CrossRef] [PubMed]
- Lucarotti, C.; Oddo, C.; Vitiello, N.; Carrozza, M. Synthetic and Bio-Artificial Tactile Sensing: A Review. Sensors 2013, 13, 1435–1466. [Google Scholar] [CrossRef]
- Spigler, G.; Oddo, C.M.; Carrozza, M.C. Soft-neuromorphic artificial touch for applications in neuro-robotics. In Proceedings of the 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy, 24–27 June 2012; IEEE: New York, NY, USA, 2012; pp. 1913–1918. [Google Scholar] [CrossRef]
- Bounakoff, C.; Hayward, V.; Genest, J.; Michaud, F.; Beauvais, J. Artificial fast-adapting mechanoreceptor based on carbon nanotube percolating network. Sci. Rep. 2022, 12, 2818. [Google Scholar] [CrossRef]
- Kerr, E.; McGinnity, T.M.; Coleman, S. Material recognition using tactile sensing. Expert Syst. Appl. 2018, 94, 94–111. [Google Scholar] [CrossRef]
- Cirillo, A.; Laudante, G.; Pirozzi, S. Tactile Sensor Data Interpretation for Estimation of Wire Features. Electronics 2021, 10, 1458. [Google Scholar] [CrossRef]
- Drimus, A.; Kootstra, G.; Bilberg, A.; Kragic, D. Design of a flexible tactile sensor for classification of rigid and deformable objects. Rob. Auton. Syst. 2014, 62, 3–15. [Google Scholar] [CrossRef]
- Gao, Z.; Ren, B.; Fang, Z.; Kang, H.; Han, J.; Li, J. Accurate recognition of object contour based on flexible piezoelectric and 9piezoresistive dual mode strain sensors. Sens. Actuators A Phys. 2021, 332, 113121. [Google Scholar] [CrossRef]
- Suslak, T. There and Back again: A Stretch Receptor’s Tale. 2015. Available online: https://era.ed.ac.uk/handle/1842/10474 (accessed on 1 April 2024).
- Hosoda, K.; Tada, Y.; Asada, M. Anthropomorphic robotic soft fingertip with randomly distributed receptors. Rob. Auton. Syst. 2006, 54, 104–109. [Google Scholar] [CrossRef]
- “Somatosensation-collective term for sensory signals from the body; Neuroscience Online. (n.d.). Somatosensory Processes (Section 2, Chapter 5). The University of Texas Medical School at Houston. Available online: https://nba.uth.tmc.edu/neuroscience/m/s2/chapter05.html (accessed on 17 June 2024).
- Luo, S.; Bimbo, J.; Dahiya, R.; Liu, H. Robotic tactile perception of object properties: A review. Mechatronics 2017, 48, 54–67. [Google Scholar] [CrossRef]
- Shimizu, T.; Shikida, M.; Sato, K.; Itoigawa, K. A New Type of Tactile Sensor Detecting Contact Force and Hardness of an Object. In Proceedings of the Technical Digest. MEMS 2002 IEEE International Conference, Fifteenth IEEE International Conference on Micro Electro Mechanical Systems (Cat. No.02CH37266), Las Vegas, NV, USA, 24–24 January 2002. [Google Scholar]
- IEEE Robotics and Automation Society; Institute of Electrical and Electronics Engineers. ICRA2017: IEEE International Conference on Robotics and Automation: Program, Singapore, 29 May–3 June 2017; IEEE: New York, NY, USA, 2017. [Google Scholar]
- Bouhamed, S.A.; Chakroun, M.; Kallel, I.K.; Derbel, H. Haralick feature selection for material rigidity recognition using ultrasound echo. In Proceedings of the 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Sousse, Tunisia, 21–24 March 2018; IEEE: New York, NY, USA, 2018. [Google Scholar]
- Schmidt, P.A.; Maël, E.; Würtz, R.P. A sensor for dynamic tactile information with applications in human–robot interaction and object exploration. Rob. Auton. Syst. 2006, 54, 1005–1014. [Google Scholar] [CrossRef]
- Würschinger, H.; Mühlbauer, M.; Winter, M.; Engelbrecht, M.; Hanenkamp, N. Implementation and potentials of a machine vision system in a series production using deep learning and low-cost hardware. Procedia CIRP 2020, 90, 611–616. [Google Scholar] [CrossRef]
- Dargahi, J.; Najarian, S. Human tactile perception as a standard for artificial tactile sensing—A review. Int. J. Med. Robot. Comput. Assist. Surg. 2004, 1, 23–35. [Google Scholar] [CrossRef] [PubMed]
- Delmas, P.; Hao, J.; Rodat-Despoix, L. Molecular mechanisms of mechanotransduction in mammalian sensory neurons. Nat. Rev. Neurosci. 2011, 12, 139–153. [Google Scholar] [CrossRef] [PubMed]
- Ding, S.; Bhushan, B. Tactile perception of skin and skin cream by friction induced vibrations. J. Colloid. Interface Sci. 2016, 481, 131–143. [Google Scholar] [CrossRef] [PubMed]
- Yi, Z.; Zhang, Y.; Peters, J. Biomimetic tactile sensors and signal processing with spike trains: A review. Sens. Actuators A Phys. 2018, 269, 41–52. [Google Scholar] [CrossRef]
- Kappassov, Z.; Corrales, J.-A.; Perdereau, V. Tactile sensing in dexterous robot hands—Review. Rob. Auton. Syst. 2015, 74, 195–220. [Google Scholar] [CrossRef]
- Rahiminejad, E.; Parvizi-Fard, A.; Iskarous, M.M.; Thakor, N.V.; Amiri, M. A Biomimetic Circuit for Electronic Skin with Application in Hand Prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 2021, 29, 2333–2344. [Google Scholar] [CrossRef]
- Weng, J.; Yu, Y.; Zhang, J.; Wang, D.; Lu, Z.; Wang, Z.; Liang, J.; Zhang, S.; Li, X.; Lu, Y.; et al. A Biomimetic Optical Skin for Multimodal Tactile Perception Based on Optical Microfiber Coupler Neuron. J. Light. Technol. 2023, 41, 1874–1883. [Google Scholar] [CrossRef]
- Kerr, E.; McGinnity, T.M.; Coleman, S. Material classification based on thermal properties—A robot and human evaluation. In Proceedings of the 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), Shenzhen, China, 12–14 December 2013; IEEE: New York, NY, USA, 2013; pp. 1048–1053. [Google Scholar] [CrossRef]
- Pestell, N.; Lepora, N.F. Artificial SA-I, RA-I and RA-II/vibrotactile afferents for tactile sensing of texture. J. R. Soc. Interface 2022, 19, 20210603. [Google Scholar] [CrossRef]
- Liu, F.; Deswal, S.; Christou, A.; Sandamirskaya, Y.; Kaboli, M.; Dahiya, R. Neuro-inspired electronic skin for robots. Sci. Robot. 2022, 7, eabl7344. [Google Scholar] [CrossRef]
- Luque, N.R.; Garrido, J.A.; Ralli, J.; Laredo, J.J.; Ros, E. From Sensors to Spikes: Evolving Receptive Fields to Enhance Sensorimotor Information in a Robot-Arm. Int. J. Neural. Syst. 2012, 22, 1250013. [Google Scholar] [CrossRef]
- Li, P.; Ali, H.P.A.; Cheng, W.; Yang, J.; Tee, B.C.K. Bioinspired Prosthetic Interfaces. Adv. Mater. Technol. 2020, 5, 1900856. [Google Scholar] [CrossRef]
- MacKinnon, C.D. Sensorimotor anatomy of gait, balance, and falls. Handb. Clin. Neurol. 2018, 159, 3–26. [Google Scholar] [CrossRef]
- Hunter, J.D. Matplotlib: A 2D Graphics Environment. Comput. Sci. Eng. 2007, 9, 90–95. [Google Scholar] [CrossRef]
- McKinney, W. Data Structures for Statistical Computing in Python. SciPy 2010, 445, 56–61. [Google Scholar] [CrossRef]
- Harris, C.R.; Millman, K.J.; van der Walt, S.J.; Gommers, R.; Virtanen, P.; Cournapeau, D.; Wieser, E.; Taylor, J.; Berg, S.; Smith, N.J.; et al. Array programming with NumPy. Nature 2020, 585, 357–362. [Google Scholar] [CrossRef]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine learning in Python. J. Mach. Learn Res. 2011, 12, 2825–2830. [Google Scholar]
- Stagi, A. Nanpy Firmware. Available online: https://github.com/nanpy/nanpy-firmware (accessed on 21 May 2024).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sharma, Y.; Ferreira, P.; Justham, L. Hardness Classification Using Cost-Effective Off-the-Shelf Tactile Sensors Inspired by Mechanoreceptors. Electronics 2024, 13, 2450. https://doi.org/10.3390/electronics13132450
Sharma Y, Ferreira P, Justham L. Hardness Classification Using Cost-Effective Off-the-Shelf Tactile Sensors Inspired by Mechanoreceptors. Electronics. 2024; 13(13):2450. https://doi.org/10.3390/electronics13132450
Chicago/Turabian StyleSharma, Yash, Pedro Ferreira, and Laura Justham. 2024. "Hardness Classification Using Cost-Effective Off-the-Shelf Tactile Sensors Inspired by Mechanoreceptors" Electronics 13, no. 13: 2450. https://doi.org/10.3390/electronics13132450