Passive Auto-Tactile Heuristic (PATH) Tiles: Novel Robot-Inclusive Tactile Paving Hazard Alert System

Abstract

Mobile service robots often have to work in dynamic and cluttered environments. Multiple safety hazards exist for robots in such work environments, which visual sensors may not detect in time before collisions or robotic damage. An alternative hazard alert system using tactile methods is explored to pre-emptively convey surrounding spatial information to robots working in complex environments or under poor lighting conditions. The proposed method for robot-inclusive tactile paving is known as Passive Auto-Tactile Heuristic (PATH) tiles. These robot-inclusive tactile paving tiles are implemented in spatial infrastructure and are aimed to allow robots to pre-emptively recognize surrounding hazards even under poor lighting conditions and potentially provide improved hazard cues to visually impaired people. A corresponding Tactile Sensing Module (TSM) was used for the digital interpretation of the PATH tiles and was mounted onboard a mobile audit robot known as Meerkat. The experiment yielded a 71.6% improvement in pre-emptive hazard detection capabilities with the TSM using a customized Graph Neural Network (GNN) model.

1. Introduction

Robots are deployed in more public spaces, with the range of tasks performed by robots increasing over time [1,2,3], especially those tasks that are considered dull, dirty, and dangerous to humans [4]. In particular, mobile service robots are conducting more cleaning, cleanliness auditing [5] and inspection tasks in built-up transit zones like hotel lobbies, airports, and restaurants [6,7]. These mobile service robots compensate for the lack of manpower for mundane and tedious jobs [8,9]. Robots encounter operational inefficiency if the existing environment is not catered for their deployment, along with the spatial obstacles they can encounter. Robot stoppage, damage to property and robots through collisions, and reduced robotic productivity are some problems that occur if the situation is not addressed.

Existing robotic detection methods that rely solely on visual means are limited by occlusion, lighting conditions, and image quality [10,11]. The sensors installed on robots let them perceive their surroundings and the hazards in their workspace. Mobile robots utilize a variety of sensors for them to localize themselves within their deployed workspace [12] and move to the next point of interest based on their sensor inputs and programming. Some of the most common localization techniques involve Simultaneous Localization And Mapping (SLAM) [13], landmark recognition or beacon arrays [14], the use of fiducial markers [15,16], and the Global Positioning System (GPS) [17]. A summary of these methods and their limitations is seen in

Table 1. Typical visual localization methods used by existing mobile robot platforms.

Localization Method	Description	Limitations
SLAM [13]	Use of LIDAR sensors to generate two-dimensional maps of robot and its surroundings	- Limited to extent of sensor operational capabilities; - Compounded sensor errors over time; - Computational resources increase drastically as area being mapped increases.
Robot beacon array [14]	Usage of Wifi/RFID and other wireless technologies to help robot to triangulate its location within a given space	- Effective range limited to extent of beacon placement; - Computational resources increase drastically as more beacons are implemented in workspace.
Fiducial markers [15]	Visual (usually monochromatic) markers placed at strategic locations for robot to detect and localize	- Require line of sight; - Affected by lighting conditions.

. Error information obtained from the sensors could lead to errors in the robot’s localization or false positives in detecting the surrounding hazards, depending on the robot’s software [18,19,20].

Robots inevitably run into obstacles in their environment during operation, be it objects in their path (spatial hazards) or items that cause detection problems in the robot’s input sensors (perceptional hazards). A non-exhaustive list of spatial hazard classes encountered by mobile service robots is categorized in

Table 2. Categories of robotic indoor hazards and robots’ responses upon encounter.

Spatial Hazards to Robots	Examples	Common Responses by Robots
Static obstacles	- Doors; - Walls; - Interior furniture; - Dead-ends.	- Slow down; - Reverse; - Pause to adjust.
Moving/transient obstacles	- Human traffic; - Vehicle traffic.	- Slow down; - Stop.
Level changes/inclines	- Stairs;/escalators; - Ramps/slopes; - Elevator entrances.	- Stop; - Reverse; - Turn left/right; - Pause to adjust.
Narrow passage	- Gaps between furniture items; - Corridors; - Choke points; - Walkways.	- Slow down; - Pause to adjust; - Reverse; - Turn left/right.
Sharp turns	- Path junctions; - Corners; - Path bifurcations.	- Slow down; - Pause to adjust; - Turn left/right.
Overhangs	- Table edges; - Railings; - Spatial barriers.	- Reverse; - Turn left/right; - Stop.

. The examples listed are not exhaustive but contain the general types of hazards that mobile robots encounter during operation.

Robot designers should consider alternative methods for hazard detection for robots to sense their environment instead of solely looking upon visual means of spatial hazard detection. In light of the listed limitations in visual means, tactile means could be explored for object localization and detection [21] and even be combined with vision sensors [22,23,24,25] for more comprehensive hazard detection of obstacles in the environment. In [26], various modified tactile sensors were experimented upon in different simulations to show the viability of using tactile sensors in robotics for hazard perception.

1.1. Advantages of Tactile Sensing Usage by Robots

There are numerous benefits to using tactile sensing in hazard detection for robots. The main advantages that tactile sensing could provide are cheaper hardware implementation [27,28] and reduced computational load compared with visual methods [29,30].

The use of tactile methods as sensor inputs could reduce hardware implementation costs, with less reliance on expensive visual detection equipment such as LIDAR or depth cameras. The resources used for such robotic components, such as rare metals and plastics, would be freed for other uses. By bringing down the overall costs of building robots, more robots could then be built with the same amount of financial capital for sales or deployment, improving cost effectiveness and overall production output [30].

Tactile sensing could cut down the computational load that the robotic processors operate with due to simplified data types being parsed, oftentimes by having the system detect the tactile changes upon the surface themselves as Boolean states, instead of having to deal with visual clutter, sub-optimal lighting conditions, or visual inputs with grainy image resolutions.

Visual clutter or downgraded image resolution can increase the time or computational memory required for the robot to identify and detect hazards in its surroundings. Tactile sensing could also complement visual sensing to reduce error rates in the spatial hazard detection of the robot’s surroundings [31,32] as a backup when the visual input cannot be used effectively by the robot.

Input data gathered using tactile sensing would also be unaffected by the local lighting conditions of the area. This could enable robotic aid equipped with these tactile sensors to be deployed in regions with low lux or outdoor areas illuminated with bright lights without hampering the detection of hazards within its surroundings.

There is thus the potential and opportunity to adapt this tactile paving system to aid robotic deployment to both improve usage rates and refine the type of information conveyed by the updated tactile indicators. More information on the existing human tactile paving system is detailed in the next section.

Robots could thus benefit greatly from a tactile hazard alert system and its corresponding interpretation program to have a robust tactile hazard detection method. To achieve that, we can look into existing tactile systems used for conveying information on hazards present in the environment. One of the successful existing tactile paving systems effectively aids visually impaired or blind (VIB) users in navigating to specific amenities such as elevators and service counters in public settings. It also helps VIB people avoid obstacles in their path [33]. Existing tactile sensing infrastructure is underutilized by the general population but is still needed by VIB people for their daily commute. Then, tactile sensing could also enable robots to be deployed in areas with extreme lighting conditions or in restricted areas that prohibit visual mapping.

1.2. Existing Tactile Paving System for Visually Impaired/Blind People

The tactile paving system was created by Seiichi Miyake in 1965 and implemented in Okayama, Japan, in 1967 [34]. Standards for the design and implementation of tactile paving are referenced from ISO/FDIS 23599 [35] and CEN/TS 15209 [36]. Countries such as Singapore [37,38], New Zealand [39], the United Kingdom (UK) [40], the United States [41], and Japan [42] have their own customized national standards for tactile paving [43,44].

These tactile paving tiles are usually brightly colored in red or yellow for distinct color contrast for locations near level changes [45,46,47] or can be colored to blend into the surrounding floor material [46]. Tactile paving is commonly made from durable materials, such as steel, granite, or hard rubber. This allows it to withstand constant foot traffic; heavy rolling loads; extreme outdoor UV levels; and other weather elements, such as wind and rain.

The existing tactile paving tiles commonly use round bumps or long, slender directional bumps [48,49]. There are specific patterns of tactile paving that convey different information types. One design provides directional cues for VIB people to guide them towards amenities. Another design denotes warnings of nearby level changes. Other designs of tactile paving tiles serve to delineate the boundary between cycling tracks and pedestrian walkways located beside each other. The distance between tactile paving and the actual hazard is dependent on the country’s requirements [41,44]. The various designs are detailed below.

Blister tactile paving tiles are typical tactile ground surface indicators (TGSIs) that have rows of flat-topped blisters placed in a rectilinear grid pattern, as seen in Figure 1A. This type of tactile paving are usually located at pedestrian crossings. They alert VIB users about the interface between footpaths and roadways or about locations with crossing gaps. The blisters are raised, truncated cones. Each blister is about 5 ± 0.5 mm high and 35–50 mm away from the others. The blisters’ base diameter is about 30–35 mm, with a diameter of 25 mm at the top.

Offset blister tiles are a specific type of blister tactile paving that warns of platform edges or cliffs ahead. The blisters are laid out in a diamond grid pattern instead of the rectilinear grid of the normal blister tiles, as seen in Figure 1B. The tile’s blisters are spaced about 66.5 mm from each other. These tiles are usually set back about 50 cm from platform edges to inform VIB users about the hazard ahead to let them stop or slow down before the level’s edge.

Corduroy tactile paving tiles are made of flat-topped bars with rounded ends to convey directional cues. These bars are 6 ± 0.5 mm high and about 20 mm in width and are spaced 50 mm apart, as shown in Figure 1C. The length of the raised bars is dependent on the tile’s length and any local tactile paving standards. Corduroy paving aligned parallel to the direction of travel denotes a safe path for VIB users to follow along. Corduroy paving tiles that are laid perpendicularly to the direction of travel are often located near level changes (such as staircase landings, ramps, level crossings, and intersections of footpaths and roadways) to warn nearby users of the presence of hazards ahead.

Lozenge paving tiles are a subtype of corduroy tactile paving that functions as a platform-edge warning surface, with rows of 6 ± 0.5 mm rounded lozenge shapes, as seen in Figure 1D. These lozenge tiles are arranged parallel to the platform edge and set back from the platform edge to give VIB users time to stop and change direction once they come into contact with lozenge tactile paving.

Cycleway tactile paving is made of multiple, continuous, raised, flat-topped stripes. Each stripe is 5 ± 0.5 mm high, about 30 mm wide, and spaced 70 mm apart. The central delineator strip is 12–20 mm high, 150 mm wide, 5 cm in width with sloping sides. The delineator strip is usually colored in white for visual contrast for sighted cyclists. Cycleway tactile paving is located on any shared route where the pedestrian side is not physically separated from the cyclist side. They are often located at the end of a shared segregated route, at regular intervals along its length, and at junctions intersecting with other pedestrian or cyclist routes.

Directional or guidance tactile paving tiles are a series of raised, flat-topped bars running in the direction of pedestrian travel. The bars are 5–6 mm in height, 35 mm wide, and spaced 45 mm apart. They are implemented in zones as obstacle avoidance or directional guides to guide VIB people to amenities.

Directional and blister tiles are often used in combination, such as those seen in Figure 2, to communicate a safe path to VIB people in public areas with multiple spatial hazards. At junctions where there is a directional change, a 2 × 2 grid of four blister tiles is usually used to indicate that there is a fork in the tactile path for VIB users. Drop-off points and pedestrian crossings also require blister tactile paving that spans the pavement–road interface.

An overview of the various tactile paving types and functions is shown in

Table 3. Summary of existing tactile paving types.

Tactile Indicator Type	Typical Locations	Usage/Function
Blister paving	Before crossing interfaces	Alerting VIB users of intersection ahead and to proceed with care
Offset blister paving	Near level change of railway platform edge	Warning VIB users of platform edges/level drop
Corduroy hazard warning paving	Near obstacles or level changes	Warning VIB users of hazards ahead, to proceed with care
Lozenge paving	On-street platform edges	Warning VIB users of platform edge of light transport systems
Cycleway paving	Beginning and end of cycleway and pedestrian intersections	Alerting VIB users of pedestrian pathways and cycleway paths
Guidance/ directional paving	Safe route around obstacles	Identifying safe route for VIB users, providing directional cues, and avoiding obstacles

, referenced from [43,48]. Most countries implement blister and directional tiles more for widespread usage compared with the other tile variants.

Current tactile paving only provides limited tactile information on the surrounding spatial hazards to VIB people. More could be added to such a system to enhance the information conveyed to both humans and robots on the types of spatial hazards in the vicinity. Information such as the type of hazard and how far it is from the current tactile paving could be encoded upon the modified tactile paving. Examples could be warning VIB users and robots of level changes like stairs and slopes, or zones with high traffic, and the corresponding distance from the tactile paving’s location. Some existing tactile paving structures have installation errors and differences in applications among countries [50,51], which leads to improper or unsafe paths [52] for VIB users. Consequences of such counterproductive tactile aid placement could be directing VIB people into obstacles planted in the way or guiding them to places with numerous fast-moving obstacles, leading to collisions and injuries. Having an updated tactile paving catalog and proper installation layouts could help provide a way of correcting these errors by implementing the updated system.

1.3. Modifying Ground Tactile Indicators as Hazard Warning System for Robots

Upgrading the tactile paving catalog would be beneficial for both VIB people and mobile robots to passively obtain data on the surrounding hazards and obstacles, and their corresponding distances, with minimal processing or the need for wireless network connectivity in the area. This improved tactile paving system would also require less maintenance than active hazard warning systems and could also be scaled to outdoor environments without being subject to extreme lighting conditions.

The working zones of mobile robots could be spatially denoted by placing boundary tiles at the perimeter of the workspace to alert humans and robots working simultaneously in the same zone. The tiles could be installed on floor surfaces instead of requiring the invasive embedding of materials used for detection within the floor slabs during the building stage, making them easier to install or replace. This novel, exploratory design for an updated tactile paving system is termed Passive Auto-Tactile Heuristic (PATH) tiles and will be discussed in its own section.

This paper proposes contributions in the following areas:

• The usage of modified ground tactile markers used as an advanced hazard warning system for mobile robots, known as Passive Auto-Tactile Heuristic (PATH) tile system.
• A designed low-cost Tactile Sensor Module (TSM) made to detect such tactile cues.
• A customized graph neural network (GNN) framework made for robots to robustly interpret the tiles and for informing the robot about the hazards ahead.

Among the following sections of the paper, Section 2 delves into the novel modified tactile paving tiles, known as Passive Auto-Tactile Heuristic (PATH) tiles, and the designed TSM and expands on GNNs and dataset generation for allowing robots to interpret PATH tiles using the TSM. Section 3 discusses the results and validation of the GNN model on the PATH tiles inputted by the TSM. Section 4 discusses the accuracy of the GNN in real-life experiments and discusses future works. Section 5 then concludes the paper.

2. Materials and Methods

The upgraded tactile paving system, also known as Passive Auto-Tactile Heuristic (PATH) paving tiles, is a supplemental hazard warning system adapted to conveniently help mobile service robots move safely through their work environment. The modified tactile hazard detection system does not compromise on its existing function of providing hazard recognition or spatial guidance to VIB users but builds upon the existing system for VIB users and includes mobile service robots as users for them to discern and navigate their environment with a lower likelihood of accidents.

Customizing the existing tactile system, which is integrated worldwide in architectural settings, could help in applicability and ease of implementation to improve the productivity of robot deployment, akin to passive infrastructural elements such as integrated roofing for adjustable shading and ventilation [53,54] or passive building cooling systems [55,56] to improve working conditions for people. There are various obstacles faced by mobile robots during operation; hence, PATH tiles could help define the different types of spatial hazards and relay the hazard data to the robots’ sensors for differentiated interpretation and response. This tactile sensing system could thus enhance the robustness of robotic environmental hazard detection and act as a fail-safe system for robots’ operational safety by employing tactile infrastructure for robotic hazard detection.

With the modified tile designs, a proper interpreter system is also developed to allow robots to decipher the information from the PATH tiles installed in their environment. This is achieved by using a customized TSM and its related Graph Neural Network (GNN). The GNN enables the robot to parse multiple separate parameters and embed relationships, as well as catering to the need fir quick prediction through noise reduction during operation.

2.1. Passive Auto-Tactile Heuristic (PATH) Tiles

PATH tiles are a catalog of modified tactile paving tiles that convey distance and hazard data to robots working in the area. The implementation of PATH tiles thus aims to build upon a novel approach [57,58,59] by improving robotic productivity with the use of robot-inclusive architectural design changes, instead of solely improving the robots’ functionalities.

The proposed infrastructural system with PATH tiles is not adversarially created as another hazard warning system, such as those implemented in existing built-up areas, but is a novel, non-intrusive architectural integration. This system aims to improve mobile robots’ ability in detecting and reacting to environmental hazards in their deployment sites. Implementing this modified tactile system could also provide an impetus for correcting installation errors in existing tactile paving layouts. The PATH tiles are categorized into two types, one for the hazard type (hazard PATH tiles) and the other for conveying the distance from PATH tiles to an obstacle (distance PATH tiles), as seen in Figure 3 for differentiated responses to different hazard placement. When discussing the distance PATH tiles, it is assumed that the robot takes its current location upon coming into contact with the PATH tile as the corresponding distance from the hazard.

A typical PATH tile measures 300 mm by 300 mm and has reflectional symmetry along the axis of travel to allow the robot to decode the same information when the robot traverses the PATH tile. Counterparts to existing tactile indicators, as seen in the bump designs of the guidance tile, the moving obstacle tile, and the vertical level change PATH tile, were designed to be analogous to the existing guidance tile, blister tile, and offset blister tile designs, respectively. This consistency in the signals conveyed could aid VIB users in interpreting the updated tactile paving. This was reflected in the design choices, whereby truncated blisters are still used to mainly indicate hazards and raised bars or stripes are used to indicate paths of travel in the updated tactile paving system [41]. This maximizes the overlap with the functions of some of the existing tiles and the subsequent information conveyed.

The split between distance and hazard PATH tile designs would give robots the time to detect and process the information conveyed by the PATH tiles when they travel over the tiles at different speeds. It also allows customization between the various safety distances and hazard types based on different object layouts for different areas.

Distance PATH tiles are designed with horizontal bars or bumps, with the density of bars or bumps increasing when the tile denotes hazards at a closer distance ahead of the robot and the TSM. Hazard PATH tiles have a varied design in terms of bump patterns to allow the TSM to make a better distinction among the various hazard types ahead.

The distance and hazard PATH tiles are meant to be used in conjunction to provide a complete representation of the hazard type and its distance from the current position of the robot as the robot passes over the PATH tiles. For blister and guidance tiles, being analogs of existing tactile paving, they can be arranged in a similar manner to convey the same type of information on which path to follow. The PATH tile pair is arranged perpendicularly to the direction of travel, spanning the width of the path if needed, to mitigate cases where a robot does not detect the PATH information due to gaps in the PATH tile layout.

For our experiment, the tiles were fabricated through 3D printing, split into halves and quarters due to constraints in print bed size, using PLA material. Samples of the 3D-printed tiles that were used in multiples for our experiment are shown in Figure 4.

2.2. Design of a Tactile Sensor Module for PATH Tile Classification

As discussed in Section 1, the detection of PATH tiles using visual means is hindered by limitations such as lighting conditions, image quality, and occlusion. As such, the perception capabilities of the robot through visual equipment, like stereo cameras, may not be the most appropriate option, as the detection of hazards could be masked by environmental conditions. Therefore, given the benefits of including tactile methods for hazard detection discussed in Section 1.1, we propose the development of a novel TSM that collects information from the PATH tiles using non-visual means. The elegance of this tactile detection system is that the contact switches do not use light or sound as a form of input, rendering it insusceptible to changes in lighting or auditory conditions. However, being a tactile sensor that is dependent on ground contact, it is sensitive to ground surface undulations. This is mitigated using the Graph Neural Network to filter out irrelevant bump patterns and ensure that the tiles can be interpreted by the robot properly. Moreover, it depends on the robot’s pathing algorithm whether the robot traverses the tiles on its route for the sensor to detect the tiles in the first place. The TSM was mounted as a modular component onboard an in-house-developed autonomous mobile robot, Meerkat. Details of Meerkat are seen in Figure 5, while its system architecture is seen in Figure 6. This robot is capable of hazard detection and spatial auditing and is controlled using the Robot Operating System 2 (ROS2) software architecture.

Our proposed in-house-developed TSM aims to be modular for robots and easy to use. To satisfy these conditions, limit switches were used as the major components to detect the variation in PATH tile contours that the robot passes over. Limit switches can toggle operating states as a reaction upon crossing a threshold input value [60], making them suitable to detect changes in critical values such as pressure or voltage. The favorable advantages of reliability, precision, and ease of installation further provide easy monitoring and data collation when deployed in real-world environments. This module was further integrated with our GNN models, enabling the robot to classify PATH tiles based on readings received by the TSM.

The TSM consists of a linear array of 10 KH-9015-HRL limit switches that are evenly spaced and mounted onto the sensor frame that was designed and 3D-printed in our lab using PLA filament, as shown in Figure 7. The TSM was installed onto the front of the Meerkat, as shown in Figure 8.

Given that limit switches detect changes through Boolean logic conditions, two types of readings are produced. Each limit switch is connected through the Common and Normally Open (NO) ports. In this study, a reading of 0 indicates that the switch is depressed and an elevated contour is present at the position of that specific limit switch. In contrast, a reading of 1 indicates that the switch is not depressed and no contour is present at that specific position. Figure 9 depicts the limit switches being depressed and released as the robot traverses PATH tiles to identify the type of PATH tile pattern.

Figure 10 shows a close-up of a possible limit switch depression configuration when passing over a PATH tile from the front view. During deployment, the data collected from the engagement and disengagement of the limit switches within the TSM provide an approximate detection of the PATH tile that the robot is traversing.

To obtain the tactile data, the linear array of tactile sensors is made to traverse the PATH tile to a generate 2-dimensional (2D) tactile input array made by consecutive 1-dimensional (1D) inputs. The 2D array is then used as input for the GNN classification system to enable the robot to autonomously recognize the information provided by the PATH tiles with regards to the hazards nearby.

2.3. Graph Neural Network PATH Tile Classification Framework

GNNs have received significant coverage over recent years due to the ability to map relationships and connections among multiple complex data parameters into a consolidated graph network well, which has resulted in convincing performance. As data structures, graphs have great expressive abilities that can be used to model a multitude of applications, ranging from social networks [61] and knowledge relations [62] to even traffic forecasting [63], and were also explored for usage in structural crack detection in maintenance works [64,65,66]. Furthermore, additional features and attributes can also be integrated into the graph structure to supplement the network to provide more complexity and detailed information about the relationships.

The GNN classification framework is split into training pipeline and deployment phases, shown in Figure 11. In the training pipeline phase, the selected GNN model is trained in simulation based on a curated PATH tile dataset. The trained weights from the model are then tuned and optimized prior to deployment. During deployment, the optimized weights are loaded onto the GNN model using the same network architecture to classify PATH tiles in real-time. To model the states of the limit switches as a graph, displayed in the overview in Figure 12, the Boolean logic for each of the 10 limit switches is consolidated. A value of 0 indicates that the switch is engaged, which represents traversing an elevated contour. In contrast, a value of 1 represents that the switch is disengaged and no contour is detected. This forms an array of binary values at a particular time instance. In total, n instances of such an array are taken to form a data batch as the robot moves across the PATH tiles. This is to ensure that there are sufficient data to clearly distinguish among and classify the 8 tile patterns. A ground-truth representation of each of the PATH tiles is displayed in Figure 13, with black cells representing 0 values and white cells representing 1 values.

The sequence of the collected $10\times n$ data points forms a data batch representing a single node in the graph and is given a label tagged to the PATH tile pattern class. An edge connection is formed between two nodes if the sequence and the values are completely equivalent. To supplement graph modeling, two edge attributes were added. The first attribute is the total number of zeroes in each data batch, represented by $A_1$. This shows the number of contour elevations detected at all limit switch positions over the course of n instances. The second attribute is the sum of the number of Boolean logic value changes from each limit switch position over n instances, represented by $A_2$. For example, for each limit switch across 2 consecutive instances, a change is valid if the Boolean logic value switches between 2 states, either from 0 to 1 or from 1 to 0. In contrast, if the states remain the same between the 2 instances, no change is detected. This is repeated for all 10 limit switches on the TSM. A simplified weight score using the two attributes is also calculated. Given that both attributes are equally important in describing the characteristics of each data batch, the weight score, represented by W, is the average of the sum between the total number of zeroes and Boolean logic value changes, as shown in Equation (1). In the context of this experiment, the goal of the GNN is to predict the class of an unlabeled node when inputted into the graph using the learned embeddings and representations of labeled nodes.

$$\begin{array}{c}\hfill W=\frac{A_1+A_2}{2}\end{array}$$

(1)

For dataset curation, each ground-truth representation is rotated in both directions, mirrored horizontally or vertically, or subject to random noise addition. In practical scenarios, the robot may not always be perfectly aligned with the tiles during the data collection process for pattern detection. Additionally, some of the limit switches may also not engage or disengage with their supposed respective Boolean logic values. As such, the generated dataset arrays can cater, simulate, and compensate for deviations by the robot during locomotion while still providing the avenue for unique pattern detection and classification. The artificially introduced variations were used in the training and validation steps of GNN model tuning. This allowed the model to learn and recognize patterns similar to the 8 listed and null patterns. With the tuned GNN model, the TSM can perform consistently regardless of environmental conditions. The validation dataset is for allowing the model to recognize what is to be considered part of the 8 tiles plus null detection instances. The best-performing trained GNN model is then further tuned to refine the trained weights to cater for any of the previously unknown patterns.

A total of 10,000 data batches were generated for each PATH tile pattern in reference to the respective ground-truth representations, with a 80–20% training–validation data split. Training and validation masks were generated to ensure that the training and validation data were kept separate. In total, 3 sets of test data, comprising 2250 data batches, were generated for further testing and evaluation.

In this paper, 3 baseline GNN models were considered based on their relevance to map networks and relationships for node classification tasks, such as predicting the research topic of different articles within a citation network or the gene type within protein–protein interactions. Zhou et al. [67] and Xiao et al. [68] documented the usage of the convolutional and attention mechanism classes commonly used for node classification. Convolutional mechanisms focus on conducting pooling or convolutional operations within the graph structure. This enables the extraction of features and higher-level representation for each specific node, which can then be inputted into a node classifier. Attention mechanisms rely on learning the different contributions towards the target node as opposed to relying on a single fixed weight. The mechanism also varies the number of neighbors that the target node can take, which can thus accommodate inputs of different sizes while placing attention on specific relevant and important sections.

From the existing literature, Graph Convolutional Network (GCN) [69], Graph Attention Network (GAT) [70], and Chebyshev Graph Convolutional Network (CHEB) [71] were selected, as these models are commonly used in the analysis and prediction of unseen and unlabeled nodes. GCN is spectral-based and expresses the essential hidden representation features of each target node. GAT has the ability to capture and identify the different importance of neighborhood information for a specific target node when generating higher hidden representations. CHEB is a spectral-based graph convolutional network that has a spectral filer derived from the Chebyshev polynomial. The pipeline for training and predicting the output classes of unlabeled nodes using each of the algorithms is documented in Figure 14, and the weights of each model are saved.

The training and validation of each model were conducted on the Nuvo-8108GC Series Rev. A1 Industrial PC by Neousys Technology Inc. with the NVIDIA GeForce RTX 3080 Lite Hash Rate GPU graphics card using the Pytorch Geometric library [72]. Each model was fitted with an early stopping callback, which allows the model to stop training and validating to save computation time and memory. A threshold of 500 epochs was implemented, which instructs the model to stop training and validating. This occurs if the validation loss does not improve for 500 epochs. An additional hyper-parameter tuning process was conducted to improve the performance of the selected model. The optimized weights were then tested on the 3 test sets to demonstrate the model’s performance on unseen data in simulation.

During actual deployment, the TSM is connected to an Arduino Uno micro-controller, which is connected to the Industrial PC. The ROS2 program calls a service to run a script that obtains real-time readings of the TSM from the Arduino Uno. Upon completing data collection, the loaded GNN model and the corresponding weights classify the obtained data, after which the ROS2 service is terminated.

2.4. Experimental Setup

The experiment is a proof of concept for utilizing automated tactile infrastructure and detection systems. One of the novel aspects is the use of a GNN as a model to cater for scenarios where the robot passes over the tiles at an angle or for delayed detection during mobile robot deployment. The Meerkat robot was made to move towards the chosen hazards of stairs, ramps, and glass doors 30 times for the blister and guidance tile patterns, for both setups before and after PATH tile installation. The robot was stopped if it was going to fall off the level change or collide into the hazards. The results were then compared if they outputted the correct predicted tile patterns using the customized GNN model with the corresponding PATH tile types in the environment. The experimental setup is seen in Figure 15.

When testing with the PATH tiles, the surrounding vicinity was laid with boards to make the surface flush with the tile base to prevent readings from the initial encounter with the tiles from affecting the readings. The robot was made to pass over the tiles 10 times for the blister and guidance tile patterns, with the customized GNN used in Section 2.3 being made to differentiate between the patterns.

The obtained outcomes of the experiment are tabulated in Section 3.

3. Results

The optimized weights from the training, validation, and testing of CHEB were loaded onto the robot. In real-world deployment, the robot encounters hazards in cases where the PATH tiles are absent, due to the lack of environmental cues in case of stairs or ramps. The robot was able to detect glass using its LIDAR sensors in some of the trial runs. Stairs, ramps, and glass doors were deemed the typical hazards that would cause the most damage to the robot, the environment, and human users in the vicinity if the robot were to collide, lose balance, or fall off the edge because of the inability to detect such edges or transparent objects in time.

For the guidance PATH tile pattern, the GNN provided correct recognition in 20 out of 30 trials, whereas for the blister PATH tile pattern, the correct pattern was recognized by the GNN in 23 out of 30 trials instead, as seen in

Table 4. Results of PATH tile runs, details in Appendix C Table A1.

Detection in 30 Runs	Correct Detection	Error Detection
Blister PATH tile	20	10
Guidance PATH tile	23	7

. In the case of the blister tile pattern, incorrect predictions were made for the ’static obstacle’ or the ’2 m distance’ PATH tile patterns instead. This could have been due to the close similarities in the binary pattern created by the tiles themselves and the inherent variations in granularity of the input data in each run. This could have been due to having performed data collection at only 10 points (the number of limit switches for our current experiment) along the tile instead of a continuous line, which would have provided more distinct differences in the classification. Another possible reason for the error classification instances might be that the limit switches did not engage properly with the contours of the PATH tiles during the experiment.

From the results, we can see that installing the TSM onto the robot allowed it to better detect the upcoming hazards with the implementation of tactile aid. Given the correct detection rating of 43 out of 60 runs, there was a 71.6% success rate in pre-emptive hazard detection runs after the implementation of PATH tiles. Through this, it is seen that the tactile PATH tile patterns could be used for spatial hazard detection to improve robot safety during deployment in the future, using its corresponding GNN to utilize and differentiate the different tactile PATH tile patterns.

4. Discussion

In model training and validation, CHEB performed best in terms of the highest validation accuracy as well as the lowest validation loss, with average plateau validation accuracy and loss of 91.34% and 1.4235, respectively. Additionally, CHEB required the shortest inference time to complete training and validation before early stopping was engaged. GCN had validation accuracy of 83.86% and validation loss of 1.4456. In contrast, GAT obtained the lowest validation accuracy, 71.42%, and the highest validation loss, 1.4886, while requiring the longest training time.

Table 5. Best training and validation accuracy and loss of GNN models, the total training epochs completed before early stopping, and the total time required.

Model	Train Acc	Val Acc	Train Loss	Val Loss	Epochs	Time/s
GCN	84.20%	83.86%	1.5464	1.4456	1732	236.44
GAT	71.33%	71.42%	1.6725	1.4886	2156	663.33
CHEB	91.94%	91.34%	1.4682	1.4235	2390	93.94

collates the best loss and accuracy metrics during the training process. From our defined metrics, in the context of our task, CHEB was, therefore, selected as the most suitable model for node classification, as the model achieved the highest validation accuracy and the lowest validation loss. The loss and accuracy curves for each GNN model can be found in Appendix A.

To optimize the performance of the CHEB model, hyper-parameter tuning using the RAY library was used to provide the best configuration of hyper-parameters. The hyper-parameters used for tuning were (1) number of hidden channels, (2) dropout rate, (3) learning rate, and (4) weight decay, with the tuning range of each parameter being documented in

Table 6. Hyper-parameters used for tuning.

Hyper-Parameter	Values	Tune Type	Best Value
Hidden channels	16, 32, 64, 128	Choice	128
Dropout rate	0 to 0.5	Uniform	2.4260 × 10${}^{-1}$
Learning rate	1 × 10${}^{-4}$ to 1 × 10${}^{-1}$	Uniform	1.8055 × 10${}^{-2}$
Weight decay	1 × 10${}^{-6}$ to 1 × 10${}^{-3}$	Uniform	3.2048 × 10${}^{-6}$

. The metrics used for optimization maximized the validation accuracy, and this metric was reported back to the RAY tune reporter upon completion of every epoch. A total of 10 samples of different hyper-parameter combinations were used, and each combination was prematurely terminated if the performance of the model did not outperform the previous combinations. From the tuning process, the best values consisted of 128 hidden channels, a dropout rate of 2.4260 × 10${}^{-1}$, a learning rate of 1.8055 × 10${}^{-2}$, and weight decay of 3.2048 × 10${}^{-6}$.

The weights of the optimized model, as well as the hyper-parameters, were saved before loading for usage during implementation onto the same training and validation dataset. Upon the completion of tuning, the validation accuracy of 95.12% and the validation loss of 1.4033 were obtained with the optimized model. This improved the validation accuracy by 3.78% and reduced the validation loss by 0.0202 compared with the results obtained prior to model tuning, as shown in

Table 7. Training and validation accuracy and loss values of the CHEB model prior to and after tuning.

	Train Acc	Val Acc	Train Loss	Val Loss	Epochs	Time/s
Before	91.94%	91.34%	1.4682	1.4235	2390	93.94
After	96.07%	95.12%	1.4120	1.4033	1490	63.12

.

During the GNN baseline model training and validation phase, CHEB outperformed the other baseline models in terms of both validation accuracy and validation loss. There are a few potential reasons for CHEB’s satisfactory performance. Firstly, CHEB uses Chebyshev polynomials, which have the ability to capture information from higher-order neighborhoods. This enables CHEB to learn long-range dependencies and embeddings from nodes within a large graph effectively. Secondly, CHEB has the ability to adaptively adjust the size of the target node’s neighborhood. This allows CHEB to select and include node neighbors based on the importance of their influence and relevance to the target node. Lastly, CHEB utilizes the spectral domain for computing Chebyshev polynomials, which can lead to improved scalability and efficiency. This was advantageous in our context, given that a large-scale graph was used within the experiment.

As a means of benchmark comparison, it was difficult to find similar previous works on GNNs related to our use case. Hence, comparisons were made in reference to Convolutional Neural Network (CNN) usage for safety or hazard detection scenarios. Comparing the results of the conducted experiment with relevant state-of-the-art classification methods, [73] used a CNN model to provide early safety warnings for intrusive behavior to mitigate accidents and casualties at construction sites using video image classification. Their proposed model obtained an accuracy rate of 87% using two classes. Ref. [74] also utilized a CNN model for reducing the repair cost of damaged concrete structures by detecting and classifying the presence of bacteria within the structure. The model achieved an average validation accuracy rate of 91.5%. Ref. [75] achieved variable classification accuracy ranging from 28% to 70% across three classes with single-shot detection using CNN to streamline job site management and decision making for machine safety. The GNN provided a better neural network alternative, as it could capture more abstract relationships among data entries for our use case. CHEB achieved higher accuracy, over 90%, in our case while also being able to accommodate classification of up to eight classes compared with three classes in the comparison literature.

The saved weights of the optimized model were then loaded onto a separate CHEB model with the same architecture documented in Section 2.3. Each test set was inputted into the optimized model for evaluation, and the results of the metric scores are documented in

Table 8. Metrics of the normalized confusion matrix from the tuned CHEB model on the test sets.

Test Set	Val Acc	Precision	Recall	Specificity	F1 Score
1	96.58%	0.9660	0.9658	0.9658	0.9659
2	96.72%	0.9674	0.9672	0.9672	0.9671
3	96.76%	0.9678	0.9676	0.9676	0.9676
Average	96.69%	0.9671	0.9669	0.9669	0.9669

. From the results, an average test accuracy rate of 96.69% was obtained using the three test sets with our trained and tuned CHEB model. The obtained average test accuracy was 1.57% higher than the validation accuracy obtained after hyper-parameter tuning. The respective normalized confusion matrices for the eight different classes and the null pattern in the three test sets can be found in Appendix B.

The PATH tile system is slated to have more designs to enhance robotic wayfinding and navigation and also to demarcate different zones for the various extents of collaboration in human–robot collaborative spaces [76]. Furthermore, the TSM could be further improved with other types of tactile sensors instead of limit switches, as limit switches give inconsistent readings and could degrade over time with prolonged contact with the ground surface. The ideal case would be to use a sensor that tracks the whole span of the PATH tile to obtain an input closer to that of a line for pattern classification instead of point data. The TSM could perhaps be re-designed with adaptive sensor quantities for different robot sizes or corridor widths in different environments or by improving a support frame design for the sensor array to reduce variations in the results.

Another potential exploration direction is to improve the robustness of the PATH tile system and determine how to encode other types of environmental information onto this architectural infrastructure. Signals from the tactile detector could also be integrated into robotic systems to make use of the data gained from coming into contact with the PATH tiles in future iterations. Future ventures could also look into making the PATH tile system more conducive for usage by VIB people as well.

5. Conclusions

This paper presented the potential of using modified tactile paving known as Passive Auto-Tactile Heuristic (PATH) tiles in eight different designs as a complement in advance obstacle detection for VIB users and mobile robots. A customized TSM was developed and mounted onto a mobile robot platform for the validation of the tile-and-detector system. A validation experiment was conducted to determine the improvement in hazard detection through the implementation of spatial robot-inclusive tactile paving using PATH tiles and the corresponding TSM for enabling robots to interpret the tactile cues and showcased an improvement of 71.6% in terms of pre-emptive hazard detection by robots. This could further provide the basis for implementing infrastructural modifications to improve robotic safety in spatial environments.