Search Results (14)

There are still relatively few applications that serve the function of a traditional tactile map, allowing visually impaired individuals to explore a digital map by sliding their fingers across it. Moreover, existing technological solutions either lack a spatial learning mode or provide only limited functionality, focusing primarily on navigating to a selected destination. To address these gaps, the authors have proposed an original concept for an indoor mobile application that enables map exploration by sliding a finger across the smartphone screen, using audio spatial descriptions as the primary medium for conveying information. The spatial descriptions are hierarchical and contextual, focusing on anchoring them in space and indicating their extent of influence. The basis for data management and analysis is GIS technology. The application is designed to support spatial orientation during user interaction with the digital map. The research emphasis was on creating an effective cartographic communication message, utilizing voice-based delivery of spatial information stored in a virtual building model (within a database) and tags placed in real-world buildings. Techniques such as Text-to-Speech, TalkBack, QRCode technologies were employed to achieve this. Preliminary tests conducted with both blind and sighted people demonstrated the usefulness of the proposed concept. The proposed solution supporting people with disabilities can also be useful and attractive to all users of navigation applications and may affect the development of such applications. Full article

This study presents a wearable haptic feedback system designed to support speech training for individuals with speech and hearing impairments. The system provides real-time tactile cues based on detected phonemes, helping users correct their pronunciation independently. Unlike prior approaches focused on passive reception or therapist-led instruction, our method enables active, phoneme-level feedback using a multimodal interface combining audio input, visual reference, and spatially mapped vibrotactile output. We validated the system through three user studies measuring pronunciation accuracy, phoneme discrimination, and learning over time. The results show a significant improvement in word articulation accuracy and user engagement. These findings highlight the potential of real-time haptic pronunciation tools as accessible, scalable aids for speech rehabilitation and second-language learning. Full article

Over the last few decades, a growing number of studies have used wearable technologies, such as inertial and pressure sensors, to investigate various domains of music experience, from performance to education. In this paper, we systematically review this body of literature using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) method. The initial search yielded a total of 359 records. After removing duplicates and screening for content, 23 records were deemed fully eligible for further analysis. Studies were grouped into four categories based on their main objective, namely performance-oriented systems, measuring physiological parameters, gesture recognition, and sensory mapping. The reviewed literature demonstrated the various ways in which wearable systems impact musical contexts, from the design of multi-sensory instruments to systems monitoring key learning parameters. Limitations also emerged, mostly related to the technology’s comfort and usability, and directions for future research in wearables and music are outlined. Full article

Digital audio–tactile maps (DATMs) on touchscreen devices provide valuable opportunities for people who are visually impaired (PVIs) to explore the spatial environment for engaging in travel activities. Existing solutions for DATMs usually require extensive training for the PVIs to understand the feedback mechanism. Due to the shortage of human resources for training specialists, as well as PVIs’ desire for frequent practice to maintain their usage skills, it has become challenging to widely adopt DATMs in real life. This paper discusses the use of large language models (LLMs) to provide a verbal evaluation of the PVIs’ perception, which is crucial for the independent practice of DATM usage. A smartphone-based prototype providing DATMs of simple floor plans was developed for a preliminary investigation. The evaluation results have proven that the interaction with the LLM could help the participants better understand the DATMs’ content and could vividly replicate them by drawings. Full article

Tactile maps designed for individuals with blindness can greatly improve their mobility, safety and access to new locations. While 3D-printed maps have already been demonstrated to be a powerful tool for delivering spatial information, they might not always be available. Alternatively, a combination of audio and haptic information can be used to efficiently encode 2D maps. In this paper, we discuss the development and user-testing of a novel audio-haptic map creator application. Maps created using this application can provide people with blindness with a tool for understanding the navigational routes and layouts of spaces before physically visiting the site. Thirteen people with blindness tested various components of the virtual map application, such as audio, haptic feedback and navigation controls. Participants’ data and feedback were collected and analyzed to determine the effectiveness of the virtual maps as it relates to this user group’s readability and usability. The study showed that it was easy to use and that it efficiently delivered information about travel routes and landmarks that the participants could successfully understand. Full article

This article describes the design, implementation, and evaluation of vibrotactile concert furniture, aiming to improve the live music experience of people with hearing loss using hearing technology such as cochlear implants (CI). The system was the result of a series of participatory design sessions involving CI users with different hearing assistive setups (bi-implant, bimodal, and monoimplant), and it was evaluated in a concert scenario (drums, bass, and female vocals) at the Royal Danish Academy of Music. The project aimed to improve the music appreciation for CI users by providing a multisensory concert designed with CI challenges in mind, but not excluding normal-hearing individuals or individuals with other forms of hearing aids from participating in the event. The evaluation was based on (video-recorded) observations and postexperience semistructured interviews; the data were analyzed using event analysis and meaning condensation. The results indicate that tactile augmentation provides a pleasant experience for CI users. However, concertgoers with residual hearing reported being overwhelmed if the tactile stimulation amplitude exceeds a certain threshold. Furthermore, devices that highlight instrument segregation are preferred over ones that present a tactile mixdown of multiple auditory streams. Full article

Background: When exploring audio-tactile nautical charts without vision, users could trigger vocal announcements of a seamark’s name thanks to video tracking. In a first condition they could simply use a green sticker fastened at the tip of a finger and in a second condition they could handle a small handy green object, called the marker. Methods: In this study, we attempted to compare finger and marker tracking conditions to complete spatial tasks without vision. More precisely, we aimed to better understand which kind of interaction was the most efficient to perform either localization or estimation of distance and direction tasks. Twelve blindfolded participants realized these two spatial tasks on a 3D-printed audio-tactile nautical chart. Results: Results of the localization tasks revealed that in finger condition, participants were faster in finding geographic elements, i.e., seamarks. During estimation tasks, no differences were found between the precision of distances and direction estimations in both conditions. However, spatial reasoning took significantly less time in marker condition. Finally, we discussed the issue of the efficiency of these two interaction conditions depending on the spatial tasks. Conclusions: More experimentation and discussion should be undertaken to identify better modalities for helping visually impaired persons to explore audio-tactile maps and to prepare navigation. Full article

This paper presents an approach to enhance electronic traveling aids (ETAs) for people who are blind and severely visually impaired (BSVI) using indoor orientation and guided navigation by employing social outsourcing of indoor route mapping and assistance processes. This type of approach is necessary because GPS does not work well, and infrastructural investments are absent or too costly to install for indoor navigation. Our approach proposes the prior outsourcing of vision-based recordings of indoor routes from an online network of seeing volunteers, who gather and constantly update a web cloud database of indoor routes using specialized sensory equipment and web services. Computational intelligence-based algorithms process sensory data and prepare them for BSVI usage. In this way, people who are BSVI can obtain ready-to-use access to the indoor routes database. This type of service has not previously been offered in such a setting. Specialized wearable sensory ETA equipment, depth cameras, smartphones, computer vision algorithms, tactile and audio interfaces, and computational intelligence algorithms are employed for that matter. The integration of semantic data of points of interest (such as stairs, doors, WC, entrances/exits) and evacuation schemes could make the proposed approach even more attractive to BVSI users. Presented approach crowdsources volunteers’ real-time online help for complex navigational situations using a mobile app, a live video stream from BSVI wearable cameras, and digitalized maps of buildings’ evacuation schemes. Full article

The ubiquity of multimodal smart devices affords new opportunities for eyes-free applications for conveying graphical information to both sighted and visually impaired users. Using previously established haptic design guidelines for generic rendering of graphical content on touchscreen interfaces, the current study evaluates the learning and mental representation of digital maps, representing a key real-world translational eyes-free application. Two experiments involving 12 blind participants and 16 sighted participants compared cognitive map development and test performance on a range of spatio-behavioral tasks across three information-matched learning-mode conditions: (1) our prototype vibro-audio map (VAM), (2) traditional hardcopy-tactile maps, and (3) visual maps. Results demonstrated that when perceptual parameters of the stimuli were matched between modalities during haptic and visual map learning, test performance was highly similar (functionally equivalent) between the learning modes and participant groups. These results suggest equivalent cognitive map formation between both blind and sighted users and between maps learned from different sensory inputs, providing compelling evidence supporting the development of amodal spatial representations in the brain. The practical implications of these results include empirical evidence supporting a growing interest in the efficacy of multisensory interfaces as a primary interaction style for people both with and without vision. Findings challenge the long-held assumption that blind people exhibit deficits on global spatial tasks compared to their sighted peers, with results also providing empirical support for the methodological use of sighted participants in studies pertaining to technologies primarily aimed at supporting blind users. Full article

The Tactile Internet (TI) is an emerging area of research involving 5G and beyond (B5G) communications to enable real-time interaction of haptic data over the Internet between tactile ends, with audio-visual data as feedback. This emerging TI technology is viewed as the next evolutionary step for the Internet of Things (IoT) and is expected to bring about a massive change in Healthcare 4.0, Industry 4.0 and autonomous vehicles to resolve complicated issues in modern society. This vision of TI makes a dream into a reality. This article aims to provide a comprehensive survey of TI, focussing on design architecture, key application areas, potential enabling technologies, current issues, and challenges to realise it. To illustrate the novelty of our work, we present a brainstorming mind-map of all the topics discussed in this article. We emphasise the design aspects of the TI and discuss the three main sections of the TI, i.e., master, network, and slave sections, with a focus on the proposed application-centric design architecture. With the help of the proposed illustrative diagrams of use cases, we discuss and tabulate the possible applications of the TI with a 5G framework and its requirements. Then, we extensively address the currently identified issues and challenges with promising potential enablers of the TI. Moreover, a comprehensive review focussing on related articles on enabling technologies is explored, including Fifth Generation (5G), Software-Defined Networking (SDN), Network Function Virtualisation (NFV), Cloud/Edge/Fog Computing, Multiple Access, and Network Coding. Finally, we conclude the survey with several research issues that are open for further investigation. Thus, the survey provides insights into the TI that can help network researchers and engineers to contribute further towards developing the next-generation Internet. Full article

We present and evaluate the concept of FeelMusic and evaluate an implementation of it. It is an augmentation of music through the haptic translation of core musical elements. Music and touch are intrinsic modes of affective communication that are physically sensed. By projecting musical features such as rhythm and melody into the haptic domain, we can explore and enrich this embodied sensation; hence, we investigated audio-tactile mappings that successfully render emotive qualities. We began by investigating the affective qualities of vibrotactile stimuli through a psychophysical study with 20 participants using the circumplex model of affect. We found positive correlations between vibration frequency and arousal across participants, but correlations with valence were specific to the individual. We then developed novel FeelMusic mappings by translating key features of music samples and implementing them with “Pump-and-Vibe”, a wearable interface utilising fluidic actuation and vibration to generate dynamic haptic sensations. We conducted a preliminary investigation to evaluate the FeelMusic mappings by gathering 20 participants’ responses to the musical, tactile and combined stimuli, using valence ratings and descriptive words from Hevner’s adjective circle to measure affect. These mappings, and new tactile compositions, validated that FeelMusic interfaces have the potential to enrich musical experiences and be a means of affective communication in their own right. FeelMusic is a tangible realisation of the expression “feel the music”, enriching our musical experiences. Full article

The natural environment and our interaction with it are essentially multisensory, where we may deploy visual, tactile and/or auditory senses to perceive, learn and interact with our environment. Our objective in this study is to develop a scene analysis algorithm using multisensory information, specifically vision and audio. We develop a proto-object-based audiovisual saliency map (AVSM) for the analysis of dynamic natural scenes. A specialized audiovisual camera with 360${}^{\circ }$ field of view, capable of locating sound direction, is used to collect spatiotemporally aligned audiovisual data. We demonstrate that the performance of a proto-object-based audiovisual saliency map in detecting and localizing salient objects/events is in agreement with human judgment. In addition, the proto-object-based AVSM that we compute as a linear combination of visual and auditory feature conspicuity maps captures a higher number of valid salient events compared to unisensory saliency maps. Such an algorithm can be useful in surveillance, robotic navigation, video compression and related applications. Full article

Visually impaired people (VIP) feel a lack of aid for their facilitated urban mobility, mainly due to obstacles encountered on their routes. This paper describes the design of AudioMaps, a prototype of cartographic technology for mobile devices. AudioMaps was designed to register the descriptions and locations of points of interest. When a point is registered, the prototype inserts a georeferenced landmark on the screen (based on Google Maps). Then, if the AudioMaps position is next to (15 or 5 m from) the previously registered point, it sends by audio the missing distance and a description. For a preview, a test area located in Monte Carmelo, Brazil, was selected, and the light poles, street corners (name of streets forming the intersections), and crosswalks were registered in AudioMaps. A tactile model, produced manually, was used to form the first image of four sighted people and four VIP, who completed a navigation task in the tested area. The results indicate that both the tactile model and the audiovisual prototype can be used by both groups of participants. Above all, the prototype proved to be a viable and promising option for decision-making and spatial orientation in urban environments. New ways of presenting data to VIP or sighted people are described. Full article

This paper reports an empirical study of a multisensory map used by visually impaired primary school pupils, to study human habitats and differences between urban, suburban and rural areas using a local example. Using multimodal analysis, we propose to examine how the use of smell and taste shape pupils’ engagement and the development of a non-visual knowledge of geography. Our research questions include: How do pupils try to make sense of this unusual material, in conjunction with the tactile, audio and tangible material used in this lesson? How does the special education teacher support the development of these interpretations? Multisensory material has the potential to support experiential and embodied learning: were these promises achieved? Our findings show how this multisensory map reconfigures spatial occupation and interaction dynamics, and that it has the potential to make the classroom more pervasive to pupils’ social, spatial and emotional lives. In doing so, it provides opportunities for the teacher to develop citizenship education. The paper provides concrete examples of uses of smell and taste in learning activities to support engagement, and has implications for pedagogical design beyond special education. Full article