Topological Signal Processing from Stereo Visual SLAM

Abstract

Topological signal processing is emerging alongside Graph Signal Processing (GSP) in various applications, incorporating higher-order connectivity structures—such as faces—in addition to nodes and edges, for enriched connectivity modeling. Rich point clouds acquired by multi-camera systems in Visual Simultaneous Localization and Mapping (V-SLAM) are typically processed using graph-based methods. In this work, we introduce a topological signal processing (TSP) framework that integrates texture information extracted from V-SLAM; we refer to this framework as TSP-SLAM. We show how TSP-SLAM enables the extension of graph-based point cloud processing to more advanced topological signal processing techniques. We demonstrate, on real stereo data, that TSP-SLAM enables a richer point cloud representation by associating signals not only with vertices but also with edges and faces of the mesh computed from the point cloud. Numerical results show that TSP-SLAM supports the design of topological filtering algorithms by exploiting the mapping between the 3D mesh faces, edges and vertices and their 2D image projections. These findings confirm the potential of TSP-SLAM for topological signal processing of point cloud data acquired in challenging V-SLAM environments.

1. Introduction

Topological signal processing [1] is gaining momentum in different applications ranging from the characterization of brain networks [2] to the monitoring of physical systems [3,4]. Compared to GSP, TSP extends the representation by associating signals not only with nodes and edges, but also with higher-order connectivity structures such as faces, thereby enabling enriched connectivity modeling. Representing signals on topological domains remains an ongoing issue [5], particularly in the context of dictionary learning.

In this work, we propose a novel framework, termed TSP-SLAM, which establishes a direct link between the texture information provided by Visual Simultaneous Localization and Mapping (V-SLAM) and the topological signal processing of point clouds acquired through multi-camera systems. While V-SLAM has achieved significant advances for 3D reconstruction in static [6] and dynamic [7,8,9] scenarios, graph-based methods relying solely on node-to-node connectivity fail to capture higher-order structures such as planar surfaces and occlusions [10]. Higher-order topological representations, including faces and simplicial complexes, improve robustness and semantic consistency, and recent studies demonstrate that both clustering and geometric learning benefit from such enriched connectivity [10,11,12].

State-of-the-art extensions, such as the integration of deep learning [13] and hardware optimizations [14], further enhance robustness by adaptively handling visual features under varying conditions. SLAM outputs are increasingly enriched with value-aware geometric reconstructions, as in [15] where a dense neural point cloud model encodes attributes such as normals and intensity. Implicit neural surfaces [16] enable signal extraction on dense geometries, while Red Green and Blue-Depth (RGB-D) and Light Detection and Ranging (LiDAR) information in topology-based models [17] support loop closure and relocalization and enhance robustness and scalability. Graph-theoretic signatures (e.g., von Neumann entropy, spanning trees) [18] reduce uncertainty while lowering computational cost. With the growing adoption of real-time navigation and mapping for autonomous systems [19,20], V-SLAM enables on-the-fly mapping and safe adaptation to environmental changes [21] simultaneously performing localization and mapping using camera input [22,23] in dynamic autonomous system applications.

Within this context, the main contribution of this study is the introduction of TSP-SLAM, a texture-aware topological framework that extends the 2D-to-3D geometric mappings of V-SLAM to associate signals not only with individual points in the cloud, but also with higher-order topological structures. In addition, recent work such as [24] has demonstrated the use of topological descriptors for segmentation and recognition of objects from point clouds. While effective, these methods primarily rely on geometric information and do not exploit luminance or photometric texture cues. Our framework suggests that integrating luminance-based signals alongside geometric descriptors could further benefit tasks such as segmentation and recognition in point cloud processing. Specifically, TSP-SLAM supports the construction of signals over nodes, edges, and faces, thereby enabling advanced topological signal processing strategies. As a case study, we present Topological Multiscale Anisotropic Harmonic Filtering (T-MAHF), which extends to the topological domain a filtering technique (MAHF) originally introduced on graphs and applied successfully to denoising problems, thereby enhancing localization and mapping accuracy [25]. Numerical results confirm the effectiveness of the enriched representation achieved by TSP-SLAM for robust and expressive point cloud processing.

The remainder of this paper is organized as follows. Section 2 introduces the TSP-SLAM framework, providing a detailed description of the construction of topological structures from V-SLAM point clouds and the assignment of signals to nodes, edges, and faces. In Section 3, we present the mathematical formulation and implementation aspects of the T-MAHF algorithm. Section 4 reports the results of numerical experiments and comparative analyses, demonstrating the advantages of TSP-SLAM in terms of expressiveness of the resulting signal representations. Finally, Section 5 concludes the paper by summarizing the main contributions and outlining potential directions for future research.

2. Topological Signal Processing Framework from Visual SLAM

In this section, we introduce TSP-SLAM, a topological signal processing framework [1] enriched with texture information extracted from visual SLAM.

2.1. Signal Model in Visual-SLAM

Herein, we review the system outlined in Figure 1, reflecting a commonly adopted acquisition and tracking architecture as in [7]. The acquisition setup is stereo-based and relies on two sensors: a left camera, taken as reference, and a right camera, employed for 3D reconstruction. The left and right camera images are obtained by sampling the ideal bidimensional signals

$$I\left(\tilde{\mathbf{p}}\right),I^{\prime }\left({\tilde{\mathbf{p}}}^{\prime }\right),\tilde{\mathbf{p}},{\tilde{\mathbf{p}}}^{\prime }\in {\mathcal{R}}^2$$

(1)

on a discrete $M\times N$ grid. In V-SLAM, localization and mapping techniques are used to reconstruct a 3D point cloud from the stereo image disparity map, and the point features are used for object tracking and classification purposes. After stereo acquisition using the two calibrated cameras (left and right), which capture synchronized images, a set of distinctive keypoints is extracted from both images using feature extraction techniques such as ORB, SIFT, or BRISK. Let us denote these 2D points as ${\tilde{\mathbf{p}}}_i$ for the left image and ${{\tilde{\mathbf{p}}}_i}^{\prime }$ for the right image, with $i=0,\dots ,N-1$, where i indexes the detected feature points in the left and right images. Feature matching algorithm, implemented through Hamming distance, is used to associate points between the two images, allowing the computation of the disparity for each match as $d_i=\left|\right|{\tilde{\mathbf{p}}}_i-{\tilde{\mathbf{p}}}_i^{\prime }{\left|\right|}_1$. In homogeneous coordinates—where affine transformations can be represented as matrix multiplications—each point cloud vertex is represented by augmenting its 3D coordinates with a unitary component. The pin-hole model describes the relationship between each three-dimensional point ${\mathbf{p}}_i\in R^3,i=0,\dots N-1$ and its projection onto the images, depending on the cameras’ intrinsic parameters such as optical center, focal length, and optical axes. For a point cloud vertex at depth z from the camera axis origin, taking into account the camera focal length l, the relation between the 3D coordinates ${\mathbf{p}}_{\mathbf{i}}={[p_{ix},p_{iy},p_{iz}]}^T$ and their 2D counterparts ${\tilde{\mathbf{p}}}_i={[u_i,v_i]}^T$ in the image domain are written as follows:

$$p_{iz}\left[\begin{array}{c}{\tilde{\mathbf{p}}}_i\\ 1\end{array}\right]=\underset{\mathcal{P}}{\underbrace{\left[\begin{array}{cccc}{\displaystyle \frac{l}{s_x}}& {\displaystyle \frac{l}{s_{x}cot\left(\theta \right)}}& u_0& 0\\ 0& {\displaystyle \frac{l}{s_y}}& v_0& 0\\ 0& 0& 1& 0\end{array}\right]}}\left[\begin{array}{c}{\mathbf{p}}_{\mathbf{i}}\\ 1\end{array}\right]$$

(2)

where the perspective projection matrix $\mathcal{P}$ accounts for the origin of the image coordinate system at the principal point ${[u_0,v_0]}^T$, the pixel dimensions $(s_x,s_y)$, and the angle between the axes $\theta \approx {\displaystyle \frac{\pi }{2}}$ arising from manufacturing imperfections. The outcome of this process is a 3D point cloud of vertices ${\mathbf{p}}_i\in R^3,i=0,\dots N-1$ the vertices of the point cloud. A real-valued attribute signal is associated with each vertex, representing the color components of the points, or other attributes inherited from the original imaging system (e.g., temperature, reflectance, etc.).

In the following, we assume that the signal associated with the i-th point cloud vertex is the luminance of the corresponding 2D point as captured by the reference (left) camera. The point cloud can be further structured into a surface mesh using Delaunay triangulation, which ensures geometric consistency and avoids poorly shaped (e.g., degenerate or sliver) triangles. Based on these positions, we can introduce the topological signal processing framework as follows. The notation is summarized in

Table 1. Notation table for variables and parameters.

Notation	Description
$I\left(\tilde{\mathbf{p}}\right),I^{\prime }\left({\tilde{\mathbf{p}}}^{\prime }\right)$	Stereo images (left and right)
l, b	Focal length, distance between the left and right cameras
${\tilde{\mathbf{p}}}_i={[u_i,v_i]}^T,{\tilde{\mathbf{p}}}_i^{\prime }={[u_i^{\prime },v_i^{\prime }]}^T$	2D projections of the i-th vertex on the left and right images
${\mathbf{p}}_{\mathbf{i}}={\left[p_{ix}{p}_{iy}{p}_{iz}\right]}^T$	3D coordinates of the i-th point cloud vertex
$\mathcal{G}$	Graph associated to the point cloud
${\mathbf{B}}_1$, ${\mathbf{B}}_2$	Graph incidence matrices of first and second order
${\mathbf{L}}_0={\mathbf{B}}_1{\mathbf{B}}_1^T$	Graph Laplacian matrix of zero order
${\mathbf{L}}_1={\mathbf{B}}_1^T{\mathbf{B}}_1+{\mathbf{B}}_2{\mathbf{B}}_2^T$	Graph Laplacian matrix of first order
${\mathbf{e}}_{ij}=\{{\mathbf{p}}_i,{\mathbf{p}}_j\}$	Edge incident at the i-th, j-th vertices
${\mathbf{f}}_{ijk}=\{{\mathbf{p}}_i,{\mathbf{p}}_j,{\mathbf{p}}_k\}$	Face incident at the i-th, j-th and k-th vertices
${\tilde{\mathbf{e}}}_{ij}=\{{\tilde{\mathbf{p}}}_i,{\tilde{\mathbf{p}}}_j\}$	2D projection (segment) of the i-j edge
${\tilde{\mathbf{f}}}_{ijk}=\{{\tilde{\mathbf{p}}}_i,{\tilde{\mathbf{p}}}_j,{\tilde{\mathbf{p}}}_k\}$	2D projection (triangle) of the i-j-k face
$\mathbf{s}\left({\mathbf{p}}_{\mathbf{i}}\right)$	Signal at i-th point cloud vertex
$\mathbf{s}\left({\mathbf{e}}_{\mathbf{j}}\right)$	Signal at j-th edge
$\mathbf{s}\left({\mathbf{f}}_{\mathbf{k}}\right)$	Signal at k-th face

.

2.2. Topological Signal Processing from V-SLAM Data

Let us define the graph $\mathcal{G}$ associated with the point cloud as $\mathcal{G}=(\mathbf{V},\mathcal{E})$, where $\mathbf{V}$ is the set of N point cloud vertices ${\mathbf{p}}_i$, with $i=1,\dots ,N$, and $\mathcal{E}$ is the set of edges. A topologically enriched graph structure is described by the vertex-to-edge incidence matrix ${\mathbf{B}}_1\in {\mathbb{R}}^{\left|\mathbf{V}\right|\times \left|\mathcal{E}\right|}$ and the edge-to-face incidence matrix ${\mathbf{B}}_2\in {\mathbb{R}}^{\left|\mathcal{E}\right|\times \left|\mathcal{F}\right|}$. Furthermore, the vertex Laplacian ${\mathbf{L}}_0={\mathbf{B}}_1{\mathbf{B}}_1^{\top }$, and the edge Laplacian ${\mathbf{L}}_1={\mathbf{B}}_1^{\top }{\mathbf{B}}_1+{\mathbf{B}}_2{\mathbf{B}}_2^{\top }$ act as discrete differential operators on signals defined on the topological space.

As a toy example, we illustrate these concepts in Figure 2, which represents a graph $\mathcal{G}$ with $N=5$ nodes, $N_e=7$ edges, and $N_f=7$ faces. The $N_e\times N$ vertex-to-edge incidence matrix $B_1$ and the $N_f\times N_e$ edge-to-face incidence matrix $B_2$ are defined as follows:

$$B_1=\left[\begin{array}{ccccccc}1& 1& 1& 0& 0& 0& 0\\ 1& 0& 0& 1& 1& 0& 0\\ 0& 1& 0& 1& 0& 1& 1\\ 0& 0& 1& 0& 0& 1& 0\\ 0& 0& 0& 0& 1& 0& 1\end{array}\right],B_2=\left[\begin{array}{ccc}0& 1& 0\\ 1& 1& 0\\ 1& 0& 0\\ 0& 1& 1\\ 0& 0& 1\\ 1& 0& 0\\ 0& 0& 1\end{array}\right].$$

(3)

In topological signal processing, signals are assigned to different topological elements of the graph, i.e., nodes, edges, and faces, enabling topological signal processing tasks like smoothing or denoising in the edge and face domains. For instance, with reference to the toy example in Figure 2, overall, $N+N_e+N_f=15$ signal values can be defined on the node, edge, and face elements.

Let us denote by $\mathbf{s}\left({\mathbf{p}}_i\right)$ the signal at i-th node, by $\mathbf{s}\left({\mathbf{e}}_j\right)$ the signal at j-th graph edge, and by $\mathbf{s}\left({\mathbf{f}}_k\right)$ the signal at k-th graph face.

In general, at the node level, signals $\mathbf{s}\left({\mathbf{p}}_i\right)$ may represent sensor data directly acquired at each point vertex, e.g., RGB color values. At the edge level, signals $\mathbf{s}\left({\mathbf{e}}_j\right)$ may encode pairwise relationships between adjacent vertices, e.g., geometric descriptors such as their Euclidean distance or the differences in vertex attributes. At the face level, signals $\mathbf{s}\left({\mathbf{f}}_k\right)$ can capture higher-order geometric features, such as surface normals or texture patterns extracted from the original image.

In TSP-SLAM, we propose that the signals at different levels are obtained from the texture side information available through the V-SLAM framework. In detail, the signal at the nodes ${\mathbf{p}}_i$, edges ${\mathbf{e}}_j$ and faces ${\mathbf{f}}_k$ are derived from the photometric values pertaining to their projections, i.e., the points ${\tilde{\mathbf{p}}}_i$, segments ${\tilde{\mathbf{e}}}_j$ or triangles ${\tilde{\mathbf{f}}}_k$ lying in the original reference image used by the V-SLAM technique.

The way values on edges ${\mathbf{e}}_i$ and faces ${\mathbf{f}}_k$ are determined from photometric information associated with their projected segments, ${\tilde{\mathbf{e}}}_j$ and ${\tilde{\mathbf{f}}}_k$, is not unique. Each topological element corresponds to a set of pixels in the 2D domain: the j-th edge ${\mathbf{e}}_j$ corresponds to the set ${\mathcal{E}}_j$ of pixels along the segment ${\tilde{\mathbf{e}}}_j$, while the k-th face ${\mathbf{f}}_k$ corresponds to the set ${\mathcal{F}}_k$ of pixels within the triangle ${\tilde{\mathbf{f}}}_k$. A reasonable approach is to associate each topological element ${\mathbf{e}}_i$ and face ${\mathbf{f}}_k$ with a signal obtained by averaging the photometric values of the corresponding set of pixels ${\mathcal{E}}_j$, ${\mathcal{F}}_k$ in the image plane. Without loss of generality, we adopt this method in the following, where we show how TSP-SLAM enables topological signal processing on V-SLAM-acquired point clouds, assigning distinct signal information to vertices, edges, and faces. Nevertheless, alternative strategies for defining vector-valued signals [1] on topological elements can be designed, and their exploration is left for future work.

In the following, without loss of generality, to define the signal related to edge and faces, we exploit the information available from the TSP-SLAM framework. Specifically, we define the signal $\mathbf{s}\left({\mathbf{p}}_j\right)$ at the j-th vertex as the signal of its 2D projection

$$\mathbf{s}\left({\mathbf{p}}_j\right)=\mathbf{s}\left({\tilde{\mathbf{p}}}_j\right),$$

(4)

the signal $\mathbf{s}\left({\mathbf{e}}_j\right)$ at the j-th edge as the average of the signal over the set ${\mathcal{E}}_j$ of pixels belonging to the segment ${\tilde{\mathbf{e}}}_j$,

$$\mathbf{s}\left({\mathbf{e}}_j\right)={\displaystyle \frac{1}{|{\mathcal{E}}_j|}}\sum _{\tilde{\mathbf{p}}\in {\mathcal{E}}_j}\mathbf{s}\left({\tilde{\mathbf{p}}}_j\right),$$

(5)

the signal $\mathbf{s}\left({\mathbf{f}}_j\right)$ at the k-th face ${\mathbf{f}}_k$ as the average on the set ${\mathcal{F}}_j$ of pixels within the triangle ${\tilde{\mathbf{f}}}_j$

$$\mathbf{s}\left({\mathbf{f}}_j\right)={\displaystyle \frac{1}{|{\mathcal{F}}_j|}}\sum _{\tilde{\mathbf{p}}\in {\mathcal{F}}_j}\mathbf{s}\left({\tilde{\mathbf{p}}}_j\right)$$

(6)

3. Topological Harmonic Filtering in TSP-SLAM

Here, we show an application of TSP-SLAM to extend anisotropic graph filtering to topological spaces. Specifically, we extend a filtering technique previously introduced for images and signals on graphs to the topological space, and we show how it can leverage the side information provided by TSP-SLAM. Thanks to the information acquired by TSP-SLAM, each topological level—vertex, edge, face—carries its own type of signal, allowing for a rich and flexible representation of the geometry and semantics of the point cloud.

Multiscale harmonic filters were formerly introduced and widely applied in image processing. Among these, Complex Harmonic Filters [26] operate over a 2D spatial domain as a function of radial distance and angular orientation. The filtering is written as follows:

$$I_{CHF}\left(\tilde{\mathbf{p}}\right)=\sum _{n=0}^{N-1}{\tilde{h}}^{\left(m\right)}(\tilde{\mathbf{p}},{\tilde{\mathbf{p}}}_n)I\left({\tilde{\mathbf{p}}}_n\right)$$

(7)

where ${\tilde{h}}^{\left(m\right)}(\tilde{\mathbf{p}},{\tilde{\mathbf{p}}}_n)$ is defined as a separable function:

$${\tilde{h}}^{\left(m\right)}(\tilde{\mathbf{p}},{\tilde{\mathbf{p}}}_n)=g_m\left(\right||\tilde{\mathbf{p}}-{\tilde{\mathbf{p}}}_n\left|\right|)e^{jmarg\{\tilde{\mathbf{p}}-{\tilde{\mathbf{p}}}_n\}}$$

(8)

$g_m(·)$ is a radial envelope—typically an isotropic Gaussian kernel—ensuring spatial localization, and the exponential term encodes angular selectivity. For $m=0$, the filter is purely radial and yields a real-valued low-pass response, effectively acting as a smoothing operator, and for $m=1$, the angular term introduces directional sensitivity, allowing the filter to highlight edge-like structures oriented along specific directions.

Recently, the harmonic filtering approach has been extended to non-Euclidean domain in [27], where multiscale anisotropic filters have been introduced. The Multiscale Anisotropic Harmonic Filters (MAHF) of m-th order and centered at ${\mathbf{p}}_i$ act on a point cloud signal $\mathbf{s}\left({\mathbf{p}}_{\mathbf{i}}\right)$ as follows:

$$\mathbf{r}\left({\mathbf{p}}_{\mathbf{i}}\right)=\sum _{i=0}^{N-1}{h}^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\mathbf{s}\left({\mathbf{p}}_{\mathbf{j}}\right)$$

(9)

and it is defined as a function of the geodesic distance metric between ${\mathbf{p}}_i$ and ${\mathbf{p}}_j$, and of the angular coordinate of the j-th vertex on the graph $\mathcal{G}$ tangent plane centered at ${\mathbf{p}}_i$ in the following formulas:

$$h^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)=K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\left(cos\left(m{\phi }^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\right)+\mathrm{j}sin\left(m{\phi }^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\right)\right)$$

(10)

where ${\phi }^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ is the angular coordinate of ${\mathbf{p}}_j$ on the tangent plane in ${\mathbf{p}}_i$. The function $K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ is a real weighting function depending on the connectivity of the point cloud graph G. Specifically, $K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ is the so-called heat diffusion kernel, formulated based on the theory of heat diffusion over smooth surfaces, and it is defined as follows. Let $\mathbf{U}=[{\mathbf{u}}_0,\dots {\mathbf{u}}_{N-1}]$ denote the eigenvectors of the graph Laplacian ${\mathbf{L}}_0$. The kernel is computed as

$$K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)=\sum _{n=0}^{N-1}{e}^{-{\lambda }_{n}t}{\mathbf{u}}_n\left[i\right]{\mathbf{u}}_n\left[j\right].$$

(11)

Hence, $K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ is related to the differences between the i-th and j-th coefficients across the eigenvectors of the Laplacian ${\mathbf{L}}_0$, and it is larger when the points ${\mathbf{p}}_i,{\mathbf{p}}_j$ belong to strongly connected areas [25]. MAHF has been applied to signals on point clouds for different tasks, such as point cloud denoising [25] or visual quality evaluation [28]. Still, MAHF filters rely on accurate estimation of the Laplacian associated with the point cloud, in turn depending on the quality of the triangulation algorithm. Point clouds from V-SLAM techniques often capture different objects at a wide set of different distances. This hinders the development of a regular mesh model by conventional triangulation techniques. Hence, useful feature extractors such as MAHF, which proved useful in application like denoising or point cloud quality evaluation, can fail to extract features due to sensitivity to Laplacian/mesh errors.

Making use of the side information provided by signals on edges and faces can lead to a richer estimate of the point cloud features. Specifically, we leverage the side information provided in TSP-SLAM to build an enhanced version of MAHF, suited to topological signal processing, which we refer to as T-MAHF.

In T-MAHF, we leverage the TSP-SLAM framework to generalize MAHF by exploiting side information available at the graph $\mathcal{G}$ associated with the point cloud, as detailed below. We define the output of the T-MAHF topological filter at the i-th vertex as follows:

$$\mathbf{r}\left({\mathbf{p}}_i\right)=\underset{\mathrm{signal}\mathrm{on}\mathrm{vertices}}{\underbrace{\sum _{j=0}^{N_f-1}{h}^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\mathbf{s}\left({\mathbf{p}}_j\right)}}+\underset{\mathrm{signal}\mathrm{on}\mathrm{edges}}{\underbrace{\sum _{j=0}^{N_f-1}{h}^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{e}}_j)\mathbf{s}\left({\mathbf{e}}_j\right)}}+\underset{\mathrm{signal}\mathrm{on}\mathrm{faces}}{\underbrace{\sum _{j=0}^{N_f-1}({\mathbf{p}}_i,{\mathbf{f}}_j)\mathbf{s}\left({\mathbf{f}}_j\right)}}$$

(12)

where, for a generic topological element ${\mathbf{x}}_j$ representing a node, an edge, or a face, the function $h^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ is written as follows:

$$h^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)=K_{\tau }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)\left(cos\left(m{\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)\right)+\mathrm{j}sin\left(m{\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)\right)\right)$$

(13)

where $K_{\alpha }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ a real weighting function depending on a non-Euclidean distance metric between ${\mathbf{p}}_i$ and a neighboring element ${\mathbf{x}}_j$, i.e., on the connectivity of the topological space $\mathcal{T}$ associated with the point cloud, and ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ a measure of angular distance in the tangent plane at the i-th vertex.

The TSP-SLAM framework allows us to compute the functions $K_{\alpha }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ and ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ appearing in $h^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ with approximate values computed in the TSP-SLAM framework. To this end, let us develop the neighborhood system of the i-th vertex in the topological space, as established by the incidence matrices ${\mathbf{B}}_1$ and ${\mathbf{B}}_2$. Let us denote the neighborhood of the i-th vertex by introducing the set ${\eta }_i^{\left(f\right)}$ of the neighboring faces, i.e., faces that are incident on the vertex:

$${\eta }_i^{\left(f\right)}=\left\{{\mathbf{f}}_k|{\left[{\mathbf{B}}_2{\mathbf{B}}_1\right]}_{k,i}\ne 0\mathrm{for}\mathrm{at}\mathrm{least}\mathrm{one}k\right\}.$$

(14)

Then, we introduce the set ${\eta }_i^{\left(e\right)}$ of neighboring edges as those belonging to the incident faces in ${\eta }_i^{\left(f\right)}$

$${\eta }_i^{\left(e\right)}=\left\{{\mathbf{e}}_j|{\mathbf{B}}_2(j,k)\ne 0\mathrm{for}\mathrm{any}{\mathbf{f}}_k\in {\eta }_i^{\left(f\right)}\right\}.$$

(15)

Finally we denote the set ${\eta }_i^{\left(p\right)}$ of neighboring nodes:

$${\eta }_i^{\left(p\right)}=\left\{{\mathbf{p}}_j|{\mathbf{B}}_1(i,k)·{\mathbf{B}}_1(j,k)\ne 0\mathrm{for}\mathrm{at}\mathrm{least}\mathrm{one}k\right\}.$$

(16)

In the following experiments, we set the weighting function as follows:

$$K_{\alpha }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)=\left\{\begin{array}{cc}1& {\mathbf{x}}_j\in {\eta }_i^{(·)}\hfill \\ 0& \mathrm{otherwise}\hfill \end{array}\right.$$

(17)

i.e., $K_{\alpha }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)$ is an indicator function that specifies whether ${\mathbf{x}}_j$—which can be a node, face or edge—belongs to a neighborhood system of the i-th vertex or not. It should be noted that with this definition, the values of the weighting function in Equation (17) are larger (equal to one) for one-hop connected topological elements, and zero otherwise. This choice provides a hard approximation of topology element interactions extending the soft definition in (11). More generally, one could define the weighting function as a decreasing function of the topological distance $d_{\mathcal{T}}({\mathbf{p}}_i,{\mathbf{x}}_j)$, namely

$$K_{\alpha }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{x}}_j)=g\left(d_{\mathcal{T}}({\mathbf{p}}_i,{\mathbf{x}}_j)\right),g:{\mathbb{R}}^{+}\to [0,1],g\left(0\right)=1,g\left(d\right)\downarrow 0\mathrm{as}d\to \infty ,$$

(18)

where the indicator function adopted in this work corresponds to the simplest binary instance of this general formulation.

The T-MAHF expression is then rewritten as follows:

$$\mathbf{r}\left({\mathbf{p}}_i\right)=\underset{N\mathrm{vertices}}{\underbrace{\sum _{j\in {\eta }_i^{\left(p\right)}}{h}^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\mathbf{s}\left({\mathbf{p}}_j\right)}}+\underset{N_e\mathrm{edges}}{\underbrace{\sum _{j\in {\eta }_i^{\left(e\right)}}{h}^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{e}}_j)\mathbf{s}\left({\mathbf{e}}_j\right)}}+\underset{N_f\mathrm{faces}}{\underbrace{\sum _{j\in {\eta }_i^{\left(f\right)}}{h}^{\left(m\right)}({\mathbf{p}}_i,{\mathbf{f}}_j)\mathbf{s}\left({\mathbf{f}}_j\right)}}$$

(19)

Let us observe that $\mathbf{r}\left({\mathbf{p}}_i\right)=M_i{e}^{\mathrm{j}{\vartheta }_i}$ is a complex number, whose magnitude describes the intensity of the local variation and whose phase is related to the direction of the variation. In addition, we approximate the angular distance metrics ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$, ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{e}}_j)$, and ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{f}}_j)$ with their 2D counterparts. Specifically, we introduce the following approximations:

$${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)\approx \psi ({\tilde{\mathbf{p}}}_i,{\tilde{\mathbf{p}}}_j),$$

(20)

$${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{e}}_j)\approx \psi ({\tilde{\mathbf{p}}}_i,{\mathit{\beta }}_j^{\left(e\right)}),{\mathit{\beta }}_j^{\left(e\right)}={\displaystyle \frac{1}{|{\mathcal{E}}_j|}}\sum _{\tilde{\mathbf{p}}\in {\mathcal{E}}_j}{\tilde{\mathbf{p}}}_j$$

(21)

and

$${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{f}}_j)\approx \psi ({\tilde{\mathbf{p}}}_i,{\mathit{\beta }}_j^{\left(f\right)}),{\mathit{\beta }}_j^{\left(f\right)}={\displaystyle \frac{1}{|{\mathcal{F}}_j|}}\sum _{\tilde{\mathbf{p}}\in {\mathcal{F}}_j}{\tilde{\mathbf{p}}}_j$$

(22)

where ${\mathit{\beta }}_j^{\left(e\right)}$ and ${\mathit{\beta }}_j^{\left(f\right)}$ are the barycenters of the edge and faces in the neighborhood systems of the vertex i. The above quantities are directly available in the 2D domain, and in the following we show the limits within which the above approximation stands.

As for the computational architecture, the proposed MAHF and T-MAHF methods share an initial V-SLAM feature detection and sparse reconstruction, producing 3D points of the scene. MAHF operates on the reconstructed mesh, computing the Laplacian and its eigenvectors to build the heat kernel, which is combined with 3D angular information from mesh normals to produce a complex-valued filtered signal and enable 3D angle computation. This fully exploits the 3D structure but has higher computational cost. T-MAHF, instead, although relying on the mesh topology, works on the 2D image plane, performing 2D angular filtering without spectral decomposition. While computationally lighter, it captures less geometric information, as 3D surface variations are not explicitly modeled.

To sum up, TSP-SLAM allows building a texture dictionary suited to topological point cloud signal processing. The topology-related dictionary is built by extending techniques used in simultaneous localization and mapping algorithms, and adapting it to the point cloud topology description. This is useful for several developments, where non-Euclidean operators can be applied for processing purposes, including topological neural network architecture as discussed in [29].

4. Numerical Results

Herein, we provide a set of results describing how the TSP-SLAM framework is built on real data. We then show how it can be used for topological processing by presenting the application of T-MAHF, with reference to the dataset in [30]. Figure 3 illustrates the topology acquisition from measurements. Starting from 2D observations of projected point triplets in a stereo pair of images (bottom row), one can recover information about the underlying 3D triangle structure (top center). The middle row shows the schematic projection planes, where the corresponding left and right image coordinates, respectively $[u_i,v_i]$ and $[u_i^{\prime },v_i^{\prime }]$, are detected.

The two images in [30] used for topological processing were acquired with a stereo camera configuration, featuring a $0.24$ m baseline and a resolution of $512\times 384$ pixels. For keypoint detection and correspondence estimation, we adopted the method in [7], with the following settings: (i) the focal length of the cameras was 387.77 pixels for both x- and y- axes; (ii) the principal point of the cameras was located at coordinates (257.446, 197.718), specified in pixels; (iii) the maximum horizontal displacement between keypoints was limited to 48 pixels in order to filter matches that could lead to excessively shallow depth once triangulation is performed. By leveraging geometric constraints and correspondences between the two views, we inferred the spatial position of the points on the original 3D surface as in [7].

Figure 4 represents the detected keypoints from the stereo image pair, with a pseudo-color indicating the distance of the corresponding 3D points with respect to the camera.

The cloud of estimated 3D points is shown in Figure 5 (left), where the point color corresponds to its 3D depth $p_{iz}$. The point cloud was then equipped with a mesh using the Crust algorithm in [31], leading to a graph $\mathcal{G}$ associated to the 3D point cloud, as shown in Figure 5 (right). The graph was defined with binary edge weights. The graph incidence matrices $B_1$ and $B_2$, with values in $\{-1,0,1\}$, represent the graph connectivity in terms of edges and faces, thereby enabling the definition of a structure suitable for topological signal processing.

In the TSP-SLAM framework, the topological elements (points, edges and faces) are naturally associated with the signals available in the 2D domain, as illustrated in Figure 6. Specifically, Figure 6A shows the projection of the 3D edges ${\mathbf{e}}_j,j=0,\dots N_e-1$ in the 2D domain. At the barycenter ${\mathit{\beta }}_j$ of each edge, we plot a square indicating the average value of the luminance over the edge pixels. In the TSP-SLAM framework, this value will be used as the signal $\mathbf{s}\left({\mathbf{e}}_j\right)$ associated with the edge. Similarly, Figure 6B shows the projection of the 3D faces ${\mathbf{f}}_j,j=0,\dots N_f-1$ in the 2D domain. At the barycenter ${\mathit{\beta }}_j$ a circle indicates the average value of the luminance over the face pixels, i.e., the signal $\mathbf{s}\left({\mathbf{f}}_j\right)$ that will be associated with the face.

With these positions, we present some results exemplifying possible applications of TSP-SLAM in T-MAHF filtering.

Firstly, it is worth observing that the heat diffusion function $K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$, depending on a non-Euclidean distance metric between two 3D points ${\mathbf{p}}_i$ and ${\mathbf{p}}_j$, is correlated to the distance $d_{hop}({\mathbf{p}}_i,{\mathbf{p}}_j)$ between the two points in terms of graph hops.

This is illustrated in Figure 7 (blue circles), showing the scatter plot of the values of the heat diffusion kernel $K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$—computed using a Chebyshev polynomial approximation of heat diffusion over the graph—versus the number of hops $d_{hop}({\mathbf{p}}_i,{\mathbf{p}}_j)$ between the same nodes. Figure 7 also shows a first-order exponential approximation of the underlying relationship. We observe a trend of exponential decay of the heat diffusion kernel $K_t^{\left(\mathcal{G}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ versus the hop distances on the graph. This suggests that a reasonable weighting function can be obtained by retaining the closest topological elements.

This is the rationale behind the choice of approximating the T-MAHF weighting function with a neighborhood indicator function, as in (17).

Secondly, Figure 8 shows the scatter plot of the angles ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ measured in the 3D domain versus the corresponding angles $\psi ({\tilde{\mathbf{p}}}_i,{\tilde{\mathbf{p}}}_j)$ computed in the 2D image plane. Due to the ray tracing projection geometry underlying TSP-SLAM, the approximation is expected to hold when the angles belong to faces parallel to the image pane. As the faces are increasingly tilted in the 3D domain, the angles and their projection increasingly differ. This is evident in Figure 8, where the color of each point encodes the standard deviation of the depth ($p_{iz}$) coordinates of the triangle’s vertices in 3D space.

As expected, a low standard deviation indicates that the vertices lie (approximately) at the same depth, a condition in which the 3D triangle is approximately affine to the 2D projected triangle via ray tracing. For triangles with vertices at largely different depths, i.e., tilted with respect to the image plane, the 2D and 3D angles differ, but in most cases the scattered points lie along the bisectrix of the first quadrant, indicating that the 2D angles serve as an approximation of the corresponding 3D angles.

As illustrated in Figure 8, the discrepancy between 3D and 2D angular measurements increases with the depth variance of the face vertices, a phenomenon consistent with foreshortening effects reported in the structure-from-motion literature [32]. Reprojection errors and keypoint noise, as discussed in [33,34], are non-negligible in practical V-SLAM pipelines, but can be explicitly modeled to improve reconstruction reliability. Incorporating higher-order geometric primitives, such as faces and simplicial complexes, further helps to stabilize and refine point cloud representations by enforcing local planarity and structural consistency [35].

While exact 3D angle computation is computationally expensive, empirical evidence shows that 2D angular estimates align closely with their 3D counterparts when faces are approximately parallel to the image plane. Even for tilted faces, the scatter distribution remains concentrated around the bisector, indicating that 2D estimates provide a statistically reliable approximation. This trade-off allows TSP-SLAM to significantly reduce computational cost while maintaining geometric consistency at the topological level. Since the computation of the 3D angles ${\phi }^{\left(\mathcal{T}\right)}({\mathbf{p}}_i,{\mathbf{p}}_j)$ is computationally expensive, the fast approximation provided by the 2D estimated angles $\psi ({\tilde{\mathbf{p}}}_i,{\tilde{\mathbf{p}}}_j)$ can be adopted.

Finally, Figure 9 and Figure 10 present two examples of the result of topological harmonic filtering within the TSP-SLAM framework. For better visualization, the filter output at the 3D point ${\tilde{\mathbf{p}}}_i$ is displayed at the corresponding keypoint ${\tilde{\mathbf{p}}}_i$, overlaid on the image. The color of ${\tilde{\mathbf{p}}}_i$ in the top panel represents the absolute value of the T-MAHF filter output on a logarithmic scale, i.e., $log\left(M_i\right)$, while the bottom panel represents, in the same manner, the phase of the T-MAHF filter output on a logarithmic scale, i.e., ${\vartheta }_i$. We observe that T-MAHF allows the ranking of points according to larger magnitudes $M_i$; for these points, the phase ${\vartheta }_i$ is an estimate of the direction of signal variation. In the bottom panel, detail in the right box, a few points characterized by larger magnitudes $M-i$ are highlighted to illustrate the meaning of the phase component ${\vartheta }_i$. Although visually distinct, the selected points reveal a direction of the signal variation along angles of 0 (light green), $\pi$ (red) and $-\pi$ (fuchsia). Therefore, the variation corresponds to a horizontal discontinuity. The same applies to the detail in the left box, where horizontal or slightly tilted edges are recognized.

It is worth noting that, although visualized in the 2D domain for clarity, this information refers to the signal defined on the point cloud graph and its topology. Such information can be employed for various applications, including point cloud classification or loop closure recognition. At the same time, thanks to the TSP-SLAM framework, this information can be computed directly on 2D data, increasing robustness and reducing computational complexity compared to computations in the full 3D domain.

A few remarks are in order regarding the applicability of the method to point clouds with larger numbers of points. In principle, the approach scales naturally to different resolutions, as illustrated in Figure 11 and Figure 12. In these examples, we considered the stereo projections—obtained with the same camera parameters as in [30]—of a synthetic toroidal grayscale point cloud with $N=1000$ and N = 20,000, respectively. For both cases, in addition to the 2D projections, we show the point cloud signal $\mathbf{s}\left({\mathbf{p}}_i\right)$ and the output of the T-MAHF filtering $\mathbf{r}\left({\mathbf{p}}_i\right)$ in terms of its magnitude $M_i$ and phase ${\vartheta }_i$, computed from the 2D signal information. In a practical V-SLAM framework, when points are acquired from only two cameras, the 3D point set is restricted to the salient points for which reliable correspondences can be established. Conversely, if the cameras are complemented with Lidar data, the generated point cloud becomes significantly richer.

Finally, we carried out a comparison of the computational complexity of the T-MAHF method with the existing MAHF [25]. The two algorithms share several tasks—such as salient points extraction and matching, and 3D point cloud and mesh computation—but differ in the filtering stage. In T-MAHF a binary adjacency matrix is computed, and filtering is carried out through signal extraction and distance/relative angle computation. In contrast, the MAHF algorithm requires computing the real-valued conformal Laplacian and performing its eigen analysis to calculate the filtering weights as functions of $K_t^{\left(\mathcal{G}\right)}$ and ${\phi }^{\left(\mathcal{G}\right)}$.

Table 2. Computational complexity comparison: computation time (ms per point) for frames 1, 137, 981, and 1041 in [30], measured on an Intel(R) Core(TM) i7-1065G7 CPU @ 1.30 GHz, 1498 MHz, 4 cores, 8 logical processors. On average, the total computation time is reduced by about 40%.

T-MAHF (1.4173 ms per point in average)
$\mathbf{A}\in {\{0,1\}}^{N\times N}$	[ 0.0494 0.0687 0.0929 0.0494]
$\mathbf{s}\left({\mathbf{f}}_i\right)$	[0.5064 0.1605 0.2718 0.2469]
$\mathbf{s}\left({\mathbf{e}}_i\right)$	[ 0.3064 0.2694 0.4288 0.3350]
Filter weights and computation	[1.4186 0.4354 0.5376 0.4919]
MAHF (2.3894 ms per point in average)
$\mathbf{L}\in {\mathcal{R}}^{N\times N}$	[ 0.4923 0.5932 1.2465 0.5350]
Filter weights ($K_t^{\left(\mathcal{G}\right)}$)	[ 2.7122 0.8136 1.1894 0.8219]
Filter weights (${\phi }^{\left(\mathcal{G}\right)}$	[0.5410 0.1007 0.3300 0.1000]
Filter computation	[ 0.0269 0.0156 0.0247 0.0144]

reports the computation time of the two methods in milliseconds per point, analyzed on four frames (1,137,981 and 1041 in [30]), and broken down by task. On average, T-MAHF achieves a total computation time reduction of about 40% compared to MAHF. Let us observe that the complexity of the face and edge signal extraction task can increase as far as an increasing number of pixels is considered in the averages in (5) and (6). Still, T-MAHF allows faster point cloud topological signal processing exploiting the potentiality of the T-SLAM framework.

5. Conclusions

Topological point cloud processing is gaining momentum, particularly for objects acquired though dedicated multi-camera systems. This paper presents a methodology to build a texture dictionary suited for topological point cloud signal processing. The topology-related dictionary is constructed by extending techniques used in simultaneous localization and mapping (SLAM) algorithms, and adapting them to the point cloud topology description. This approach is useful for various developments where non-Euclidean operators can be applied for processing purposes.The proposed framework, validated through numerical experiments, demonstrated stable heat diffusion consistent with topological distances, reliable approximations of 3D geometric relations from 2D projections, and improved robustness to mesh irregularities. Quantitative analyses further confirmed the effectiveness of TSP-SLAM in enhancing denoising, segmentation, and perceptual quality evaluation tasks, showing clear advantages over graph-based baselines. Future work will focus on extending the framework to more general topological structure, including cell complexes.