Multipurpose Smart Shoe for Various Communities ^†

Abstract

A recent survey depicts that across the globe there are nearly 36 million visually impaired people facing serious issues in accessibility, education, navigating public spaces, safety concerns, and mental health. In recent times, the evolutions of obstacle detectors for blind people have been from peoples’ use of sticks, smart glasses, and smart shoes. Among the above, the major problem faced by all blind people is to walk independently to every place, so to make them feel independent while they walk, herein is a proposal for an intelligent shoe. The proposed intelligent shoe consists of a controller connected with an ultrasonic sensor, voice alert system (VAS), vibration patterns, GPS navigation, connectivity with a smart phone or smart-watch, voice assistance, feedback on gait and posture, and emergency features that are embedded with each other to communicate the presence of obstacles in the directions of the path of the blind. The sensor identifies an obstacle in the direction present then it passes the signal to the controller that activates the VAS and the vibration patterns present in that direction. Therefore, by the proposed concept of vibration sense and VAS with GPS navigation, connectivity with a smart phone or smart-watch means the system provides easy access for the blind to identify obstacles present in their way and help them toward social inclusion.

1. Introduction

Blindness is a defect that is divided into two types: one is caused by hormones inherited from the previous generation, which is labeled as permanent blindness, and the other one is caused by higher usage of radiation, which is called temporary blindness to humans. The system of network connections that is present in-between electronic components and humans is represented as the Internet of Things (IoT), where the electronic components transmit signals to humans by means of data. The detection of an obstacle, pothole, or slippery surface, as well as health tracking and heat sensing, is performed using an Arduino board. This work includes GPS-GSM navigation and location tracking, as well as providing an emergency SOS facility to assist blind people [1]. A hardware component [2] is incorporated in the IOT technology to help people with vision deficiency to solve the issues they face every day. A smart shoe for the blind is proposed in [3] which conveys the presence of obstacles in vibrational form through the use of vibrators. Here, the shoe [4] not only conveys the presence of an obstacle but also notifies the wearer of the presence of underground holes in the form of an audio message. In [5], the designed shoe has two modules, namely shoe and phone modules: the shoe module consists of a controller and sensors, whereas the phone module is linked to a GPS system. The controller is embedded with a Bluetooth transceiver which is linked to a smart phone app that uses Google maps to direct the blind to the required destination by audio message.

In [6], the system embedded in the shoe detects the presence of an obstacle present at a certain distance and conveys the information either through a vibrator or audio message, and through the use of an alarm sound, the blind can detect the presence of an obstacle. An assistance system proposed in [7] uses computer vision algorithms which are written with certain functions such as obstacle detection, avoidance, and navigation. Here, the obstacle detection is performed by sensors present, and the presence is conveyed using smart phone audio which gives haptic feedback to the blind. In [8], an intelligent shoe is embedded with sensors, a buzzer, and a microcontroller. After the detection of an object, the buzzer indicates the presence of an obstacle, and for better accuracy, smart glass is used. In [9], three pairs of ultrasonic sensors placed on the medial, central, and lateral part of the shoe are used to detect the presence of holes and pits; the study includes the usage of piezoelectric transducers for recharging the battery while walking. A wearable shoe in [10] is used to detect the presence of obstacles, wet floors, and patient falls by using voice alarms. An additional method is to prevent accidents, falling, and getting lost in some areas unknown to the visually impaired by communicating this information to the care-taker via a mobile application. With respect to the above studies, some feasible solutions have been presented for blind people all around the world. There is the usage of detection in either audio or vibration form for the transmission of messages relating to the detection of obstacles:

(i)

The usage of a GPS module to communicate the location of the visually impaired person.

(ii)

The usage of smart glass for the further detection of obstacles.

In connection with this, herein is a design of an intelligent shoe that transmits the detection of obstacles in both vibration and audio form that is embedded within the microcontroller. Upon the detection of an obstacle, the alert mode, either by voice or vibration, is passed to the blind to avoid the upcoming obstacle. Further, the design is embedded with a GPS module which locates the blind, so as to transmit the location to the care-taker in emergency situations.

2. Design of Smart Shoe Model

By the usage of an ultrasonic sensor combined with a pre-programmed Arduino controller, it is easy to detect the object present before visually impaired ones. The Arduino controller is coded in such a way that after detection of an object, the controller passes the message to the vibrator and VAS. After receiving the command from controller, the object detection is represented by means of vibration through a vibrational motor and voice alert through a speaker attached to it. Using IoT technology in the shoe that is being worn by the visually impaired gives them the independence to walk wherever they like. This paper describes the proposal of a smart shoe for the blind that indicates obstacles in the directions such as in the front, left, right, and upper part of the visually impaired making them more independent to walk along the path without depending on others. There is also an ultrasonic sensor present in the upper part of the breadboard/shoe which is used to detect the obstacle that is present in the upper view of the blind and allows those visually challenged to realize that there is obstacle. The components used for the making of this model are a battery, motor, ultrasonic sensor, Arduino board, vibrator, and a breadboard/shoe. If there is any detection of obstacles in the fixed path of the ultrasonic sensor, then the ultrasonic sensor passes the received signal to the Arduino board. After receiving the signal feedback from the ultrasonic sensor, the pre-programmed Arduino sends or transmits the signals to the vibrator and VAS. The received signal from the Arduino board makes the vibrator vibrate and instructs the VAS to interact with the user and inform about the obstacle. By this method, it is easy to help the blind to detect where the object is and avoid it so that he/she can walk freely in the road. By using the GPS module along with the shoe, the location of the blind person can be shared with the care-taker in emergency conditions. By using the CNN algorithm in the camera module of the shoe, the name of the upcoming obstacle can be mentioned to the blind person, and the distance between them and the obstacle can be accurately determined. Figure 1 represents the block diagram of the smart shoe. Figure 2 shows the simulation model in the OFF state whereby if the distance of the ultrasonic sensor is less than the fixed range, the light source is turned OFF. Figure 3 shows the simulation model in the ON state whereby if the distance of the ultrasonic sensor is greater than the fixed range, the light source is turned ON to indicate the presence of an obstacle. In the ON state, the ultrasonic sensors work based on the following order:

(i)

If the obstacle is identified in the first ultrasonic sensor, then the vibrational motor on the right side glows up (turns on) and vice versa.

(ii)

If the obstacle is identified in the second ultrasonic sensor, then the vibrational motor present at the second position on the right side glows up (turns on).

(iii)

If the obstacle is detected in the third ultrasonic sensor, then the vibrational motor present at the second position on the left side glows up (turns on).

(iv)

Finally, if an obstacle is detected at the fourth ultrasonic sensor, then the vibrational motor present on the left side glows on (turns on).

This is how the stimulation model works, so based on this functionality, the obstacle present at each direction can be identified, and based on the vibration caused by the vibrational motor, the object detection can be performed.

3. Prototype of Smart Shoe

The schematic diagram Figure 4a is sketched for the circuit connection of the prototype model, and for the proceeding of the prototype model, the workflow diagram shown in Figure 4b is designed. The prototype model is shown in Figure 5 and consists of four ultrasonic sensors with a range of distance coverage between 2 cm and 300 cm. Here, the four ultrasonic sensors are used to detect the presence of obstacles in the directions such as the following: in the right, left, straight, and upward direction of the blind person, four vibrators, four speakers, and the process of obstacle detection will take place in the following pattern: As shown in Figure 6a, if the obstacle is identified in the first ultrasonic sensor which is present in the forward direction, then the vibrational motor on the front side vibrates, and also, the VAS present on the front side produces an alert which indicates the presence of an obstacle.

As shown in Figure 6b, if the obstacle is identified in the second ultrasonic sensor present on the upside, then the vibrational motor present at the upside position vibrates, and the VAS present on the upside produces an alert sound indicating the presence of an obstacle in the upside direction. As shown in Figure 7a, if the obstacle is detected in the third ultrasonic sensor present on the right side, then the vibrational motor present on the right side vibrates, and also, the VAS present on the right produces an alert which indicates the presence of an obstacle on the right side.

As shown in Figure 7b, if an obstacle is detected at the fourth ultrasonic sensor present on the left side of the blind, then the vibrational motor present on the left side vibrates, and also, the VAS present on the left side produces an alert which indicates the presence of an obstacle in the left direction.

Thus, the entire working model of the smart shoe is explained here, so based on this, the presence of an obstacle in either direction mentioned can be identified, and based on the vibration caused by the vibrational motor and the alert from the VAS, the message can be conveyed to the visually impaired person. As shown in Figure 8, the prototype model is embedded with the GPS module in order to pass the location of the blind person in emergency conditions. Here, the SPI protocol is used with the Arduino board for communicating the location of the blind person to the care-taker.

Once the location of the blind person is transmitted to the care-taker, the output of the location will be as shown in Figure 9. The location of the blind person is communicated with the care-taker in emergency conditions (i.e., the blind will be provided with a switch to transmit the location with the care-taker) in order to make them feel free and to monitor the current location of the blind person without being with them.

4. Conclusions

In the olden days, blind people suffered from the problem of walking independently in roads, which brought about the invention of the walking stick, which was used for the detection of objects. However, as it could not sense the presence of an obstacle, there came the invention of an additional ultrasonic sensor to detect the presence of an object, but this was not as effective as first thought because the stick could not be used in every location where the blind travel. So, in order to solve this problem, presented here is IOT-based technology used in the shoe of a blind person which will help to them to walk independently in roads and help them to feel independent without depending on the help of others to walk. This helps blind people effectively as there are four ultrasonic sensors combined with the vibrators and VAS which can predetermine the presence of object in all directions when a blind person walks in the road. In this solution, there is the usage of both the VAS and vibrators, because if the blind person also has a hearing impairment, then the vibration is enough for him to detect the location of an obstacle present. By the usage of a GPS module, the blind person’s location is transmitted to the care-taker for reasons of safety in emergency situations. The placement of the ultrasonic sensors on the shoe will be as follows: two ultrasonic sensors on the right shoe to detect obstacles on the right side and in the forward direction, and two ultrasonic sensors in the left shoe for the detection of obstacles in the upward and left directions of the blind person. The prototype model is to be installed on the shoe in order to make the product more efficient for the blind to use, but due to size compactness, the model is to be reduced to a smaller size and developed within the shoe in the future.