Smart System with Artificial Intelligence for Sensory Gloves
, Claudio Guerra 3, Pedro Pagola 2,4, Blanca Bujanda 2,4 and José Javier Astrain 2,5,*Round 1
Reviewer 1 Report
A smart system with artificial intelligence for sensory gloves
This paper presents a new sensory glove that includes a gyroscope and three accelerometers to allow the selection of the appropriate signal time windows recorded by the microphone of the glove. The signal is subsequently analyzed by a convolutional neural network, which indicates whether the connection of the components has been made correctly or not.
This paper aims to describe a new sensory system capable of carrying out the verification 42 of the correct embedding in the cable connection that is done in many production lines, with success 43 rates of almost 100%.
The article is written well, and it is easy to follow. The article is reporting a fabrication of sensory gloves used in an industrial environment. Of-the-Shelf deep learning method has been used for the recognition task.
The article has no scientific novelty. Instead, it is proposing a solution for the real and industrial environment.
General Comments:
Please improve the quality of your figures (Fig.3,4,8, 9,..) It is very difficult to read the axis.
Introduction:
The connection of the wiring is made using a click that generates a characteristic sound that the operator must detect, thus ensuring that the connection (clicking) has been made correctly.
R: I think it is also possible to use gloves with tactile sensors which are more robust against noise.
Related works:
The use of smart gloves is not new at all. As described in [1], “hand movement data acquisition is used in many engineering applications”. The use of sensory gloves has been considered for many purposes, such as sign language recognition [2,3], hand posture monitoring [4,5], computer-generated (typically virtual reality or augmented vision) environments [6], tactile sensing
Please cite: A Review of Tactile Information: Perception and Action Through Touch
IEEE Transaction on Robotics 2018, And Tactile-based manipulation of deformable objects with the dynamic center of mass 2016 IEEE-RAS 16th International Conference on Humanoid Robots. AND:
Tactile-based active object discrimination and target object search in an unknown workspace Autonomous Robots 2019, And Also Robust tactile descriptors for discriminating objects from textural properties via artificial robotic skin
IEEE Transactions on Robotics 2019
force sensing for biomedical purposes [9-11], fitness exercises tracking [12], Sensing Finger Tapping in 62 Piano Playing [13], teleoperation [14], rehabilitation [15-17], and many others.
System Architecture:
Please give more information about the sensors. Are you using 3-Axis Accelerometry?
In-Page 8,
Line 253-262:
Please re-write your derivation. Please use Equation format rather than integrating mathematical derivation into text.
Please: re-write lines: 279-297
Please improve the language of the paper.
Author Response
Point 1: This paper presents a new sensory glove that includes a gyroscope and three accelerometers to allow the selection of the appropriate signal time windows recorded by the microphone of the glove. The signal is subsequently analyzed by a convolutional neural network, which indicates whether the connection of the components has been made correctly or not.
This paper aims to describe a new sensory system capable of carrying out the verification of the correct embedding in the cable connection that is done in many production lines, with success rates of almost 100%.
Response 1: Indeed, the aim of this article is to show that it is feasible to use convolutional neural networks (CNN) to improve the quality of industrial processes. The article shows that it is possible to achieve accuracy rates close to 100% in the recognition of vehicle dashboard component connecting processes, minimising the number of non-conformities during the manufacturing process. This is achieved by using sensory gloves equipped with accelerometers, gyroscope and microphone and a CNN system trained for this purpose. The system here presented has been developed on a real assembly line.
Point 2: The article is written well, and it is easy to follow. The article is reporting a fabrication of sensory gloves used in an industrial environment. Of-the-Shelf deep learning method has been used for the recognition task.
The article has no scientific novelty. Instead, it is proposing a solution for the real and industrial environment.
Response 2: We are very grateful that the reviewer found our article easy to read and well written. As the reviewer rightly points out, the contribution lies in showing the feasibility of using the system described in a complicated industrial environment, with noise levels up to 90dB without any signal processing nor noise cancelation algorithm that increases de price and complexity of the electronic of the glove. The article shows the feasibility of using CNNs to solve the problem of component connection recognition, as well as how to proceed with the training of the networks and the results of their use.
Point 3: General Comments: Please improve the quality of your figures (Fig.3,4,8, 9,..) It is very difficult to read the axis.
Response 3: We have tried to improve the quality of the images to improve the readability of the axes. Thank you for the recommendation.
Point 4: Introduction:
The connection of the wiring is made using a click that generates a characteristic sound that the operator must detect, thus ensuring that the connection (clicking) has been made correctly.
R: I think it is also possible to use gloves with tactile sensors which are more robust against noise.
Response 4: As the reviewer rightly points out, tactile sensors can be used to recognise the correct connection of components, and this is even a good option to address the problem. In our case, and due to design requirements set by the management staff of the factory, it was established that the operators should use conventional work gloves to which some accessory could be attached, but in no case was the use of specific gloves contemplated. The rationale for this requirement is that operators' gloves wear out frequently and need to be replaced. In this context, the use of microphones and accelerometers is simpler than the integration of tactile sensors into conventional gloves.
Point 5: Related works:
The use of smart gloves is not new at all. As described in [1], “hand movement data acquisition is used in many engineering applications”. The use of sensory gloves has been considered for many purposes, such as sign language recognition [2,3], hand posture monitoring [4,5], computer-generated (typically virtual reality or augmented vision) environments [6], tactile sensing
Please cite: A Review of Tactile Information: Perception and Action Through Touch
IEEE Transaction on Robotics 2018, And Tactile-based manipulation of deformable objects with the dynamic center of mass 2016 IEEE-RAS 16th International Conference on Humanoid Robots. AND:
Tactile-based active object discrimination and target object search in an unknown workspace Autonomous Robots 2019, And Also Robust tactile descriptors for discriminating objects from textural properties via artificial robotic skin IEEE Transactions on Robotics 2019 force sensing for biomedical purposes [9-11], fitness exercises tracking [12], Sensing Finger Tapping in Piano Playing [13], teleoperation [14], rehabilitation [15-17], and many others.
Response 5: Following the reviewer's recommendation, we have included both references. Thank you for the suggestion.
Point 6: System Architecture:
Please give more information about the sensors. Are you using 3-Axis Accelerometry?
Response 6: Yes, we use 3-Axis accelerometry. The IMU used (TDK InvenSense MPU-9250) is described at Line 129.
Point 7: In-Page 8, Line 253-262: Please re-write your derivation. Please use Equation format rather than integrating mathematical derivation into text.
Response 7: We have re-written our derivation following the indications of the reviewer. Thank you very much.
Point 8: Please: re-write lines: 279-297
Response 9: We have re-written the lines indicated. Thank you.
Point 9: Please improve the language of the paper.
Response 9: We have tried to improve the language of the article. We hope we have succeeded.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The authors of this manuscript propose the use of convolutional neural networks (CNN) to detect the proper connection of equipment on the dashboard of vehicles.
- Usually a large number of training samples is required to well train a CNN. As shown in Table 2, the number of samples for training CNN is several thousand, which is relatively lower than the required values. I suggest you to use sample data augmentation techniques to make more samples. Data augmentation involves the process of creating new data points by manipulating the original data. This process increases the diversity of the data available for training models in deep learning without having to actually collect new data.
- Please show the simulation curves of your CNN during training. For example, how the prediction accuracy changes with epochs, how the training loss changes with epochs.
One possible reference paper that includes these simulation curves:
Q. Huang, "High-Precision Quality Inspection for Screws Using Artificial Intelligence Technology“， CIB W78 (International Council for Research and Innovation in Building and Construction)， 2019.
Author Response
Point 1: The authors of this manuscript propose the use of convolutional neural networks (CNN) to detect the proper connection of equipment on the dashboard of vehicles.
- Usually a large number of training samples is required to well train a CNN. As shown in Table 2, the number of samples for training CNN is several thousand, which is relatively lower than the required values. I suggest you to use sample data augmentation techniques to make more samples. Data augmentation involves the process of creating new data points by manipulating the original data. This process increases the diversity of the data available for training models in deep learning without having to actually collect new data.
Response 1: As the reviewer rightly points out, the use of sample data augmentation techniques allows obtaining a great number of samples, and then to better train and fitting the networks. This is what prompted us to use synthetic samples, as shown in Tables 4, 5 and 6. In our case, we have focused on synthetic samples obtained at the laboratory because of mobility restrictions due to the pandemic (COVID'19). Following the reviewer's recommendation, we have built new samples from the real samples obtained on the assembly line (plant) and re-trained the networks. The results obtained are described on Section 6.
Point 2:
- Please show the simulation curves of your CNN during training. For example, how the prediction accuracy changes with epochs, how the training loss changes with epochs. One possible reference paper that includes these simulation curves:
- Huang, "High-Precision Quality Inspection for Screws Using Artificial Intelligence Technology“，CIB W78 (International Council for Research and Innovation in Building and Construction)， 2019.
Response 2: We thank the reviewer for his/her valuable comment and the example he/she has provided. We have explained the training process on page 13 (see Figure 14).
Author Response File: Author Response.pdf
