SnapStick: Merging AI and Accessibility to Enhance Navigation for Blind Users
Abstract
1. Introduction
1.1. Technology and Assistive Aids
1.2. Electronic Travel Aids
1.3. AI-Powered Systems
2. Methodology
2.1. Participants
2.2. SnapStick
2.3. Data Collection
- 1.
- How satisfied are you with the overall performance of the navigational aid (mobile app, cane, and bone-conduction headphones)? Answer choices included: Very satisfied, Satisfied, Neutral, Dissatisfied, and Very dissatisfied.
- 2.
- How comfortable did you feel using the navigational aid? Answer choices included Very Comfortable, Comfortable, Neutral, Uncomfortable, and Very Uncomfortable.
- 3.
- How confident did you feel navigating the environment using this aid? Answer choices included Very Confident, Confident, Neutral, Unconfident, and Very Unconfident.
- 4.
- How clear was the feedback provided through the bone-conduction headphones? Answer choices included Very Clear, Clear, Neither Clear nor Unclear, Unclear, and Very Unclear.
- 5.
- Did you encounter any problems or flaws while using the navigational aid? Answer choices included No or Yes.If yes, please describe the issue.
2.4. Data Privacy and Security
- All images captured through the app are temporarily stored in volatile memory and are automatically deleted immediately after processing.
- No images are saved to the device’s storage or transmitted externally.
- Communication between the client app and local server uses secure protocols (e.g., HTTPS or local loopback with TLS).
- No personally identifiable information (PII) is recorded, and no logging of visual content takes place.
3. Results
System Performance Evaluation
- Recognition Accuracy: We manually labeled and compared app-generated descriptions for 40 test images (including indoor scenes, images with texts, outdoor scenes, and images containing people). Descriptions were evaluated for correctness by 2 independent human raters using predefined criteria. SnapStick achieved a recognition accuracy of 94%.
- Inference Latency: The average end-to-end time (from image capture to audio feedback delivery) was 1.7 ± 0.2 s per image in our local server setup. Latency remained below 2 s in 96% of cases.
- Power Consumption: For a 30-min navigation session, the app consumed approximately 9.8% of battery on a fully charged 4500 mAh phone, corresponding to an estimated 198 mW power draw. Most of this came from camera use and VLM inference.
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
VLM | Vision Language Model |
SUS | System Usability Scale |
References
- Riazi, A.; Riazi, F.; Yoosfi, R.; Bahmeei, F. Outdoor difficulties experienced by a group of visually impaired Iranian people. J. Curr. Ophthalmol. 2016, 28, 85–90. [Google Scholar] [CrossRef] [PubMed]
- Manduchi, R.; Kurniawan, S.; Bagherinia, H. Blind guidance using mobile computer vision: A usability study. In Proceedings of the 12th international ACM SIGACCESS Conference on Computers and Accessibility, Orlando, FL, USA, 25–27 October 2010; pp. 241–242. [Google Scholar]
- Lin, B.S.; Lee, C.C.; Chiang, P.Y. Simple smartphone-based guiding system for visually impaired people. Sensors 2017, 17, 1371. [Google Scholar] [CrossRef] [PubMed]
- Zhao, Y.; Kupferstein, E.; Tal, D.; Azenkot, S. “ It Looks Beautiful but Scary” How Low Vision People Navigate Stairs and Other Surface Level Changes. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, Galway, Ireland, 22–24 October 2018; pp. 307–320. [Google Scholar]
- Thaler, L.; Goodale, M.A. Echolocation in humans: An overview. Wiley Interdiscip. Rev. Cogn. Sci. 2016, 7, 382–393. [Google Scholar] [CrossRef] [PubMed]
- Srikulwong, M.; O’Neill, E. Tactile representation of landmark types for pedestrian navigation: User survey and experimental evaluation. In Proceedings of the Workshop on Using Audio and Haptics for Delivering Spatial Information via Mobile Devices at MobileHCI 2010, Lisbon, Portugal, 7–10 September 2010; pp. 18–21. [Google Scholar]
- Holbrook, M.C.; Koenig, A.J. Foundations of Education: Instructional Strategies for Teaching Children and Youths with Visual Impairments; American Foundation for the Blind: New York, NY, USA, 2000; Volume 2. [Google Scholar]
- Sánchez, J.; Espinoza, M.; de Borba Campos, M.; Merabet, L.B. Enhancing orientation and mobility skills in learners who are blind through video gaming. In Proceedings of the 9th ACM Conference on Creativity & Cognition, Sydney, Australia, 17–20 June 2013; pp. 353–356. [Google Scholar]
- Bhowmick, A.; Hazarika, S.M. An insight into assistive technology for the visually impaired and blind people: State-of-the-art and future trends. J. Multimodal User Interfaces 2017, 11, 149–172. [Google Scholar] [CrossRef]
- Khenkar, S.; Alsulaiman, H.; Ismail, S.; Fairaq, A.; Jarraya, S.K.; Ben-Abdallah, H. ENVISION: Assisted navigation of visually impaired smartphone users. Procedia Comput. Sci. 2016, 100, 128–135. [Google Scholar] [CrossRef]
- Csapó, Á.; Wersényi, G.; Nagy, H.; Stockman, T. A survey of assistive technologies and applications for blind users on mobile platforms: A review and foundation for research. J. Multimodal User Interfaces 2015, 9, 275–286. [Google Scholar] [CrossRef]
- Ran, L.; Helal, S.; Moore, S. Drishti: An integrated indoor/outdoor blind navigation system and service. In Proceedings of the Second IEEE Annual Conference on Pervasive Computing and Communications, Orlando, FL, USA, 14–17 March 2004; pp. 23–30. [Google Scholar]
- Cuturi, L.F.; Aggius-Vella, E.; Campus, C.; Parmiggiani, A.; Gori, M. From science to technology: Orientation and mobility in blind children and adults. Neurosci. Biobehav. Rev. 2016, 71, 240–251. [Google Scholar] [CrossRef] [PubMed]
- Ferrand, S.; Alouges, F.; Aussal, M. An electronic travel aid device to help blind people playing sport. IEEE Instrum. Meas. Mag. 2020, 23, 14–21. [Google Scholar] [CrossRef]
- Bujacz, M.; Skulimowski, P.; Strumillo, P. Sonification of 3D scenes using personalized spatial audio to aid visually impaired persons. In Proceedings of the International Community for Auditory Display 2011, Budapest, Hungary, 20–23 June 2011. [Google Scholar]
- Filipe, V.; Fernandes, F.; Fernandes, H.; Sousa, A.; Paredes, H.; Barroso, J. Blind navigation support system based on Microsoft Kinect. Procedia Comput. Sci. 2012, 14, 94–101. [Google Scholar] [CrossRef]
- Kupferstein, E.; Zhao, Y.; Azenkot, S.; Rojnirun, H. Understanding the use of artificial intelligence based visual aids for people with visual impairments. Investig. Ophthalmol. Vis. Sci. 2020, 61, 932. [Google Scholar]
- Granquist, C.; Sun, S.Y.; Montezuma, S.R.; Tran, T.M.; Gage, R.; Legge, G.E. Evaluation and comparison of artificial intelligence vision aids: Orcam myeye 1 and seeing ai. J. Vis. Impair. Blind. 2021, 115, 277–285. [Google Scholar] [CrossRef]
- Association, W.M. World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. JAMA 2013, 310, 2191–2194. [Google Scholar] [CrossRef]
- Sauro, J.; Lewis, J.R. Quantifying the User Experience: Practical Statistics for User Research; Morgan Kaufmann: San Francisco, CA, USA, 2016. [Google Scholar]
- Kuriakose, B.; Shrestha, R.; Sandnes, F.E. Tools and technologies for blind and visually impaired navigation support: A review. IETE Tech. Rev. 2022, 39, 3–18. [Google Scholar] [CrossRef]
- Shafique, S.; Setti, W.; Campus, C.; Zanchi, S.; Del Bue, A.; Gori, M. How path integration abilities of blind people change in different exploration conditions. Front. Neurosci. 2024, 18, 1375225. [Google Scholar] [CrossRef] [PubMed]
- Gonzalez Penuela, R.E.; Collins, J.; Bennett, C.; Azenkot, S. Investigating use cases of ai-powered scene description applications for blind and low vision people. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 11–16 May 2024; pp. 1–21. [Google Scholar]
- Rao, S.U.; Ranganath, S.; Ashwin, T.; Reddy, G.R.M. A Google glass based real-time scene analysis for the visually impaired. IEEE Access 2021, 9, 166351–166369. [Google Scholar]
Subject | Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Total Score |
---|---|---|---|---|---|---|---|---|---|---|---|
S1 | 4 | 1 | 5 | 3 | 5 | 1 | 5 | 1 | 5 | 1 | 92.5 |
S2 | 4 | 1 | 5 | 4 | 5 | 3 | 5 | 1 | 5 | 1 | 85 |
S3 | 5 | 3 | 3 | 2 | 4 | 1 | 4 | 2 | 4 | 1 | 77.5 |
S4 | 1 | 3 | 5 | 1 | 4 | 1 | 4 | 1 | 4 | 1 | 77.5 |
S5 | 5 | 1 | 5 | 1 | 5 | 2 | 5 | 1 | 5 | 1 | 97.5 |
S6 | 5 | 1 | 5 | 1 | 4 | 1 | 5 | 1 | 5 | 1 | 97.5 |
S7 | 5 | 1 | 5 | 3 | 5 | 1 | 5 | 1 | 5 | 1 | 95 |
S8 | 5 | 1 | 4 | 4 | 4 | 1 | 4 | 2 | 3 | 1 | 77.5 |
S9 | 4 | 4 | 2 | 5 | 3 | 3 | 4 | 4 | 2 | 5 | 35 |
S10 | 5 | 1 | 5 | 2 | 5 | 1 | 5 | 1 | 5 | 1 | 97.5 |
S11 | 5 | 1 | 5 | 1 | 5 | 1 | 5 | 1 | 5 | 1 | 100 |
Subject | Q1 | Q2 | Q3 | Q4 | Q5 |
---|---|---|---|---|---|
S1 | Very Satisfied | Very Comfortable | Very Confident | Very Clear | No |
S2 | Very Satisfied | Comfortable | Confident | Very Clear | No |
S3 | Satisfied | Comfortable | Unconfident | Very Clear | No |
S4 | Neutral | Comfortable | Confident | Very Clear | No |
S5 | Very Satisfied | Very Comfortable | Very Confident | Very Clear | No |
S6 | Very Satisfied | Comfortable | Very Confident | Very Clear | No |
S7 | Very Satisfied | Comfortable | Very Confident | Very Clear | No |
S8 | Satisfied | Comfortable | Confident | Clear | No |
S9 | Satisfied | Neutral | Unconfident | Clear | No |
S10 | Satisfied | Very Comfortable | Very Confident | Clear | No |
S11 | Very Satisfied | Very Comfortable | Very Confident | Clear | No |
Feature | SnapStick | Seeing AI | Lin’s App | Gonzalez Penuela App | Rao System |
---|---|---|---|---|---|
Functionality | Scene description, text reading, facial expression, person describing, bus route recognition | Reading text, recognizing products and people, describing scenes, identifying currency | Scene description and rough distance calculation | Scene description | Scene description |
Audio Delivery | Bone-conduction (open-ear) | Standard headphones or phone speaker | Standard headphones or phone speaker | Standard headphones or phone speaker | Bone-conduction transducer |
Hands-Free Operation | Yes (via cane button) | No | No | No | Yes (voice-activated) |
Offline Capability | Partial (local server) | No (cloud-based) | Partial (local server) | No (cloud-based) | No (cloud-based) |
Qualitative Performance | SUS: 84.7 (A+)/90.9% user satisfaction | 43% satisfaction | 77.5% satisfaction | 55.2% satisfaction | Not mentioned |
Quantitative performance | 94% Accuracy | Not mentioned | 55% mAP | 65% Accuracy | 84% mAP |
Cost | Not publicly available | Free | Not publicly available | Not publicly available | Not discussed |
Privacy | Local processing | Cloud processing | Local processing | Cloud processing | Cloud processing |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shafique, S.; Bailo, G.L.; Zanchi, S.; Barbieri, M.; Setti, W.; Sciortino, G.; Beltran, C.; De Luca, A.; Del Bue, A.; Gori, M. SnapStick: Merging AI and Accessibility to Enhance Navigation for Blind Users. Technologies 2025, 13, 297. https://doi.org/10.3390/technologies13070297
Shafique S, Bailo GL, Zanchi S, Barbieri M, Setti W, Sciortino G, Beltran C, De Luca A, Del Bue A, Gori M. SnapStick: Merging AI and Accessibility to Enhance Navigation for Blind Users. Technologies. 2025; 13(7):297. https://doi.org/10.3390/technologies13070297
Chicago/Turabian StyleShafique, Shehzaib, Gian Luca Bailo, Silvia Zanchi, Mattia Barbieri, Walter Setti, Giulio Sciortino, Carlos Beltran, Alice De Luca, Alessio Del Bue, and Monica Gori. 2025. "SnapStick: Merging AI and Accessibility to Enhance Navigation for Blind Users" Technologies 13, no. 7: 297. https://doi.org/10.3390/technologies13070297
APA StyleShafique, S., Bailo, G. L., Zanchi, S., Barbieri, M., Setti, W., Sciortino, G., Beltran, C., De Luca, A., Del Bue, A., & Gori, M. (2025). SnapStick: Merging AI and Accessibility to Enhance Navigation for Blind Users. Technologies, 13(7), 297. https://doi.org/10.3390/technologies13070297