AFSMWD: A Descriptor Flexibly Encoding Multiscale and Oriented Shape Features

Abstract

Shape descriptors are extensively used in shape analysis tasks such as shape correspondence, segmentation and retrieval, just to name a few. Their performances significantly determine the efficiency and effectiveness of subsequent applications. For this problem, we propose a novel powerful descriptor called Anisotropic Fractional Spectral Manifold Wavelet Descriptor (AFSMWD), built upon an extended manifold signal processing tool named Anisotropic Fractional Spectral Manifold Wavelet (AFSMW), which is also presented for the first time in this paper. The novelty of AFSMW is integrating the fractional theory into the common anisotropic spectral manifold wavelet. Compared to the existing wavelets, it provides one more new parameter, namely, the fractional order, to balance or enhance the transform coefficients among different shape vertices, enabling more flexible local shape analysis and more hidden shape structural information explored. Due to the advantages of this added parameter and the capability of analyzing shape features from multiple scales and orientations, the AFSMW allows us to construct the powerful descriptor AFSMWD just using the AFSMW transform coefficients of a very simple function. The proposed descriptor appears to be especially localizable, discriminative, and robust to noises. Extensive experiments have demonstrated that our descriptor has outperformed the state-of-the-art descriptors, nearly achieving 22% improvements to the most related work ASMWD and 69% to the recent popular work WEDS on the FAUST dataset. Its superiorities are also announced in some challenging occasions such as shapes with large deformation or topological partiality.

1. Introduction

Building shape descriptors is a fundamental problem in the field of 3D computer vision and computer graphics. It aims to construct a vector for each shape point that can informatively describe the shape’s local geometric properties. This task has countless downstream applications such as shape correspondence, labeling, segmentation, and retrieval, among others, and the qualities would greatly affect the efficiency and effectiveness of those applications.

Over the past few years, many researchers have devoted themselves to providing efficient and robust point descriptors [1]. Among them, the spectral descriptors [2,3,4], namely, the descriptors built upon the eigenvalues (spectrum) and the eigenfunctions of the Laplace–Beltrami Operator (LBO), have achieved especially outstanding successes. They obtain many appealing properties such as invariance under shape isometric deformation, high efficiency, and noise robustness. If treated from another more in-depth viewpoint, as the eigenvalues and the eigenfunctions of the LBO, respectively, work as the frequencies and Fourier bases for signals defined on manifolds [5], the construction of major spectral descriptors can be uniformed to use harmonic analysis tools to perform certain signal processing. For instance, the famous spectral descriptors Heat Kernel Signature (HKS) [2] and Wave Kernel Signatur (WKS) [3] are based on signal filtering using either only low-frequency or band-frequency filters, while the approach [6] integrates the Windowed Fourier Transform for learning shape descriptors. Apparently, the choice of the manifold signal processing tools plays a significantly important role in their construction. Among plenty of such existing tools, the Spectral Graph Wavelet (SGW) and the Transform (SGWT) [7] show the most notable superiorities in analyzing shape information, allowing to extract multiscale shape features using both a variety of low-pass and band-pass frequency filters. Compared to the other tools, it permits to derive more desirable descriptors like the wavelet energy decomposition signature (WEDS) [4]. Moreover, by integrating oriented filters into the SGWT, ref. [8] improves the SGW’s ability to extract shape features further. They introduce direction-sensitive wavelets, i.e., the anisotropic spectral manifold wavelet (ASMW) and transform (ASMWT), enabling a comprehensive analysis of shape features from multiple local diffusion directions via several different oriented wavelets. Based on this improved wavelet tool, the Anisotropic Spectral Manifold Wavelet Descriptor (ASMWD) [8] and the Anisotropic Wavelet Energy Decomposition Descriptor (AWEDD) [9] were proposed, both showing outstanding discriminativity and the ability of intrinsic symmetry identification. Motivated by these developments in the literature, we believe that the descriptor performance could be improved further if more powerful manifold signal processing tools are introduced into the shape analysis field.

In this paper, we propose the Anisotropic Fractional Spectral Manifold Wavelet (AFSMW) and transform for shape analysis, and what is more, using this tool, we provide a novel discriminative and robust shape descriptor called Anisotropic Fractional Spectral Manifold Wavelet Descriptor (AFSMWD). The core idea is integrating the fractional order theory into the spectral manifold wavelets. As provided with one more hyperparameter, namely, the fractional order $\theta$ to control the rotation angle of the transform in the vertex-frequency plan, the wavelet is able to search for the optimal expression domain not only along the dimension of scale but also along the fractional order dimension. Therefore, it provides a larger expression space and is more likely to obtain a better expressive domain for the original problem. Meanwhile, to address the drawback of common wavelets being insensitive to the local diffusion direction of shapes, which generally is a vital factor in building informative and intrinsic-symmetry disambiguated descriptors, we build our fractional wavelets based on the eigensystem of the fractional anisotropic LBO, which is also an extension of the anisotropic LBO [10]. This means that various local diffusion directions in the underlying shape can be integrated into the wavelet transform just by tuning two added parameters, which respectively control the anisotropic degree and the orientations of the wavelets. Finally, based on the coefficients resulting from performing a series of AFSMW transforms with different shapes and orientations on a simple function, i.e., the Delta function, we propose our new descriptor named AFSMWD.

We summarize our main contributions as follows:

• We propose the AFSMW and transform, which is an extended wavelet that allows us to perform multiscale wavelet transform from various directions around each shape point, even with different rotation angles in the vertex-frequency plan. The fractional order contributes to enhancing the transform coefficients among different shape vertices and revealing more hidden shape features, making the state-of-the-art manifold wavelet ASMW [8] just act our one special case.
• We propose a powerful descriptor, AFSMWD, which is verified to be more discriminative and localized than existing descriptors while maintaining lots of desirable properties such as isometric deformation invariance and robustness to shape noises.
• Extensive experiments to evaluate the proposed descriptor are conducted. The results demonstrate that the AFSMWD outperforms state-of-the-art descriptors in shape-matching tasks. It achieves significant improvements in benchmark performance.

2. Related Work

Extensive efforts have been devoted to designing descriptors for several different shape analysis tasks. Readers can refer to a recent survey [1] for a more comprehensive understanding. Meanwhile, harmonic analysis tools, especially wavelets on graphs or manifolds, have also developed rapidly. In the following, we will review the most related works.

Shape descriptors. Spatial approaches. Early spatial domain-based descriptors are typically developed using histograms, such as integral volume descriptors [11], shape distributions [12], and multiscale local features [13]. The Signature of Histograms of Orientations (SHOT) [14] is designed by aggregating the normal angles from key points and their neighboring points. Subsequent efforts focused on constructing descriptors that are invariant to non-rigid deformations. For instance, ref. [15] proposed a new method for constructing a local reference frame (LRF) by calculating the intrinsic gradient of a scalar field on the input shape. Geodesic distance descriptors (GDDs) [16] relied on the geodesic distance basis to derive features. Nevertheless, they are highly dependent on local information and particularly sensitive to surface discretization.

Spectral approaches. Spectral descriptors have gained considerable attention recently due to their invariance to non-rigid deformations. The HKS [2] and WKS [3] were introduced, drawing inspiration from well-known physical phenomena such as heat diffusion and quantum particle evolution on surfaces. To better capture local shape characteristics, a descriptor based on the Windowed Fourier Transform (WFT) [6] was later developed. Following this, researchers introduced the Spectral Graph Wavelet Descriptor (SGWD) [17], leveraging Spectral Graph Wavelets [7] known for their multiscale and localized support, which significantly enhanced the descriptor’s discriminative power. More recently, Wang et al. [18] employed Spectral Graph Wavelets to decompose the Dirichlet energy of vertex coordinate functions, resulting in the local descriptor WEDS. The Average Mixing Kernel Signature (AMKS) [19] was derived from a quantum exploration process over the shape’s surface. A similar approach to WKS was applied to shape difference operators instead of the LBO, yielding the innovative DWKS [20]. Additionally, the Improved Biharmonic Kernel Signature (IBKS) [21] builds on the biharmonic kernel signature, maintaining beneficial properties like robustness to isometries and stability in joint regions of the shape. Lastly, the Average Increment Scale-Invariant Heat Kernel Signature (AISIHKS) [22] was designed to extract both geometric and topological features by calculating the average increment of the heat kernel across all time parameters.

While the aforementioned spectral descriptors have achieved notable success, they are based on the isotropic LBO, making them insensitive to directional information and incapable of capturing intrinsic shape symmetries. To address these limitations, researchers have focused on developing anisotropic spectral descriptors. Mathieu et al. [10] introduced the Anisotropic Laplace–Beltrami Operator (ALBO), which adjusts diffusion rates along the principal curvature directions. This innovation paved the way for a range of related works, including the Anisotropic Heat Kernel Signature (AHKS) and a descriptor built on anisotropic windowed Fourier transformation (AWFT) [23]. More recently, Li et al. [8] extended the spectral manifold wavelet framework [7] to an anisotropic setting, resulting in the creation of the ASMWD descriptor. Following this, ref. [9] applied the anisotropic spectral manifold wavelet to analyze Dirichlet energy in point coordinate functions, introducing the AWEDD descriptor. In contrast to these methods, our work expands on the more robust fractional spectral manifold wavelets by incorporating anisotropy, yielding the new AFSMWD descriptor, which demonstrates superior performance.

Deep learning approaches. With the rapid development of machine learning, lots of efforts have been made to optimize descriptors through deep learning techniques. A comprehensive understanding can be found in the survey [24]. Representative works include Optimal Spectral Descriptors (OSDs) [25] and the Anisotropic Diffusion Descriptor (ADD) [26], just to name a few. Some researchers tried to construct Convolutional Neural Networks to improve the performances of handcrafted descriptors, like Anisotropic Chebyshev Spectral CNN (ACSCNN) [27], the learned WEDS [4], the hybrid fusion network (HFN) [28], the AWEDD [9], etc. Despite the significant success of learning-based methods, they require a substantial amount of labeled data for training, which is both challenging and time intensive to obtain. Moreover, the outcomes are highly dependent on the quality of the input handcrafted descriptors. In this paper, we focus on developing a high-quality handcrafted descriptor designed to enhance the performance of both traditional handcrafted techniques and learning-based models.

Spectral harmonic analysis on manifolds. Since the eigenvalues and the eigenfunctions of the LBO, respectively, act as the frequencies and Fourier bases for signals defined on graphs or manifolds, plenty of classical signal processing tools were allowed to be extended on graphs or manifolds in recent years, such as Fourier transform [29], windowed Fourier transform [30], and Spectral Graph Wavelet Transform [7], just to name a few. On the other hand, some researchers have also devoted themselves to introducing the fractional theory into graph signal processing. Wang et al. [31,32] proposed a definition of fractional Fourier transform on graphs named graph fractional Fourier transform (GFRFT), which was based on algebraic signal processing theory. Moreover, the windowed fractional Fourier transforms on graphs were proposed in [33] to improve the GFRFT’s locality of the vertex frequency. However, GFRFT is a global transform and does not provide useful localization properties in the fractional graph domain, preventing multiresolution analysis for graph signals. Therefore, based on the spectral theory [34], Wu et al. [35] proposed the Fractional Spectral Graph Wavelets (FSGWs) to cover the shortage of GFRFT. The formerly mentioned tools can be easily generalized to analyze the signals defined on shapes (manifolds) by just replacing the graph Laplacian operator with LBO. However, only using this simple generation is still not able to produce suitable tools with sufficient geometric meaning to analyze the signals on shapes, as they are all isometric and insensitive to oriented shape features. In order to capture the shape structural information with several different diffusion directions, researchers further introduce the anisotropic FT, windowed FT [23] and wavelets [8] based on the ALBO [10]. In this work, we integrate the fractional order theory into the anisotropic manifold wavelets, which improves the abilities of the wavelets to extract shape features further.

3. Aisotropic Fractional Spectral Manifold Wavelet

In this section, we first introduce the Anisotropic Fractional Laplace–Beltrami Operator, then utilizing the eigensystems of this novel operator, we propose the Aisotropic Fractional Spectral Manifold wavelet and corresponding wavelet transforms. They are the foundation of constructing our descriptor AFSMWD.

3.1. Aisotropic Fractional Laplace–Beltrami Operator

In order to improve the shortage of the common LBO that is isometric and thus insensitive to oriented shape structural information, the researchers proposed the ALBO, which could flexibly control the local diffusion directions [10,26]. We first briefly describe this method, and then based on it, build our Aisotropic Fractional Laplace–Beltrami Operator (AFLBO).

Given a function $f\left(x\right)$ defined on the manifold $\mathcal{M}$, the tangent vectors at each point x are expressed with respect to the orthogonal basis formed by the principal curvature directions. Suppose the matrix ${\mathbf{R}}_{\theta }$ denotes a rotation around the normal by an angle $\theta$ relative to the principal curvature direction, where the origin is fixed by using the principal curvature direction in the tangent plane at point x (e.g., the direction of maximum principal curvature) as a reference. The ALBO is then defined as follows:

$${\Delta }_{\alpha \theta }f\left(x\right)=-\mathbf{div}\left({\mathbf{R}}_{\theta }{\mathbf{D}}_{\alpha }\left(x\right){\mathbf{R}}_{\theta }^{\mathrm{T}}{\nabla }_{X}f\left(x\right)\right),$$

(1)

where the thermal conductivity tensor is defined as ${\mathbf{D}}_{\alpha }\left(x\right)=\mathrm{diag}\left\{{\displaystyle \frac{1}{1+\alpha }},1\right\}$, which governs diffusion along the direction of maximum principal curvature. The parameter $\alpha$ controls the level of anisotropy in the process. The ALBO not only preserves the advantageous properties of the standard LBO but also introduces variations across multiple orientations, leading to a diffusion process that is both more intuitive and semantically meaningful.

In the discrete setting, we mainly concentrate on triangular meshes here, while the issues on other shape representations can be deduced similarly. Given a triangular mesh $\mathcal{M}$ containing N vertices, a function f defined on it is now represented as a vector $\mathbf{f}\in {\mathbb{R}}^N$. If provided with values of the anisotropic level $\alpha$ and the rotation angle $\theta$, the discrete ALBO of the mesh $\mathcal{M}$ can be represented as a sparse matrix ${\mathbf{L}}_{\alpha \theta }={\mathbf{A}}^{-1}{\mathbf{B}}_{\alpha \theta }\in {\mathbb{R}}^{N\times N},$ where $\mathbf{A}=\mathrm{diag}(a_1,\dots ,a_N)$ is the mass matrix, with $a_i$ denoting the local area element of the vertex i. Due to limited space here, detailed computation about the stiffness matrix ${\mathbf{B}}_{\alpha \theta }\in {\mathbb{R}}^{N\times N}$ is referred to the work [26]. We now compute the eigendecomposition of matrix ${\mathbf{L}}_{\alpha \theta }$ by solving the following generalized eigenproblem

$${\mathbf{B}}_{\alpha \theta }{\mathit{\varphi }}_{\alpha \theta ,k}={\lambda }_{\alpha \theta ,k}\mathbf{A}{\mathit{\varphi }}_{\alpha \theta ,k},{\mathit{\varphi }}_{\alpha \theta ,k}\in {\mathbb{R}}^N.$$

As the eigenvectors ${\left\{{\mathit{\varphi }}_{\alpha \theta ,k}\right\}}_{k=0}^{N-1}$ are A-orthogonal, the inner product of two functions $\mathbf{f}$ and $\mathbf{g}$ is computed as ${〈\mathbf{f},\mathbf{g}〉}_{\mathbf{A}}={\mathbf{f}}^{\mathrm{T}}\mathbf{Ag}$. Note that the ALBO matrix ${\mathbf{L}}_{\alpha \theta }$ is semi-definite positive. Therefore, it has non-negative and ordered eigenvalues ${\left\{{\lambda }_{\alpha \theta ,k}\right\}}_{k\ge 0}$ and a cluster of $\mathbf{A}$-orthogonal eigenvectors ${\left\{{\mathit{\varphi }}_{\alpha \theta ,k}\right\}}_{k\ge 0}$. Store the eigenvectors ${\left\{{\mathit{\varphi }}_{\alpha \theta ,k}\right\}}_{k=0}^{N-1}$ as columns of the matrix

$${\mathbf{U}}_{\alpha \theta }=[{\mathit{\varphi }}_{\alpha \theta ,0},{\mathit{\varphi }}_{\alpha \theta ,1},\cdots ,{\mathit{\varphi }}_{\alpha \theta ,N-1}]\in {\mathbb{R}}^{N\times N},$$

and the eigenvalues ${\left\{{\lambda }_{\alpha \theta ,k}\right\}}_{k=0}^{N-1}$ as a diagonal maxtrix

$${\Lambda }_{\alpha \theta }=\mathrm{diag}({\lambda }_{\alpha \theta ,0},{\lambda }_{\alpha \theta ,1},\cdots ,{\lambda }_{\alpha \theta ,\mathit{N}-1}).$$

With these notations, the anisotropic Laplacian matrix could be represented as ${\mathbf{L}}_{\alpha \theta }={\mathbf{U}}_{\alpha \theta }{\Lambda }_{\alpha \theta }{\mathbf{U}}_{\alpha \theta }^{\mathrm{T}}\mathbf{A}$. Accordingly, the anisotropic Fourier coefficient vector of the signal $\mathbf{f}$ is computed as $\widehat{\mathbf{f}}={\mathbf{U}}_{\alpha \theta }^{\mathrm{T}}\mathbf{A}\mathbf{f}$.

Now, we extend this operator to be more flexible and powerful. Given a fractional order $0<s\le 1$, we define the Anisotropic Fractional Laplace–Beltrami Operator (AFLBO). For this purpose, we first compute the fractional order of the eigenvector matrix, namely,

$${\mathbf{U}}_{\alpha \theta }^s={\left({\mathbf{U}}_{\alpha \theta }\right)}^s=[{\mathit{u}}_{\alpha \theta ,0}^s,{\mathit{u}}_{\alpha \theta ,1}^s,\cdots ,{\mathit{u}}_{\alpha \theta ,N-1}^s].$$

(2)

Meanwhile, we also compute the fractional order of the eigenvalue matrix, which is obtained by

$${\Lambda }_{\alpha \theta }^s={\left({\Lambda }_{\alpha \theta }\right)}^s=\mathrm{diag}({\lambda }_{\alpha \theta ,0}^s,{\lambda }_{\alpha \theta ,1}^s,\cdots ,{\lambda }_{\alpha \theta ,\mathit{N}-1}^s),\mathrm{where}{\lambda }_{\alpha \theta ,k}^s={\left({\lambda }_{\alpha \theta ,k}\right)}^s.$$

(3)

Finally, the AFLBO is defined as

$${\mathbf{L}}_{\alpha \theta }^s={\mathbf{U}}_{\alpha \theta }^s{\Lambda }_{\alpha \theta }^s{\left({\mathbf{U}}_{\alpha \theta }^s\right)}^{\mathrm{T}}\mathbf{A}.$$

(4)

Obviously, when $s=1$, the AFLBO becomes the common ALBO. For other values of s, we can compute ${\mathbf{U}}_{\alpha \theta }^s$ by using the Schur–Pad’ algorithm [36]. Note that, however, in practice, we generally truncate the first $K\ll N$ eigenvalues, especially in the shape analysis domain where shapes usually have thousands of vertices. Under such circumstances, the Schur–Pad’ algorithm will fail, as it only could process square matrices. Therefore, we can instead compute the SVD decomposition of ${\mathbf{U}}_{\alpha \theta }$, having ${\mathbf{U}}_{\alpha \theta }=\mathbf{P}\Sigma \mathbf{V}$, where $\mathbf{P}$ and $\mathbf{V}$ are both unitary matrices. Then, compute the fractional order of the eigenvector matrix as ${\mathbf{U}}_{\alpha \theta }^s=\mathbf{P}{\Sigma }^s\mathbf{V}$.

3.2. Anisotropic Fractional Spectral Manifold Wavelet and Transform

Based on the proposed AFLBO, we extend the wavelet and introduce the Anisotropic Fractional Spectral Manifold Wavelet and Transform, which have notable advantages for shape analysis.

Before that, we should first extend the corresponding Fourier transform as it generally acts as the basis for other more sophisticated signal processing tools. Due to the eigenvalues and the eigenvectors of the AFLBO obtaining harmonic properties, the inner product of $\mathbf{f}$ and each eigenvector ${\mathit{u}}_{\alpha \theta ,k}^s$ can be treated as an anisotropic fractional spectral manifold Fourier coefficient (transform) and play the role of frequency for signals (functions), i.e.,

$${\widehat{f}}_{\alpha \theta ,k}^s={〈\mathbf{f},{\mathit{u}}_{\alpha \theta ,k}^s〉}_{\mathbf{A}}={({\mathit{u}}_{\alpha \theta ,k}^s)}^{\mathrm{T}}\mathbf{A}\mathbf{f}.$$

(5)

Let $\widehat{\mathbf{f}}=\left({\widehat{f}}_{\alpha \theta ,k}^s\right)$ denote the Fourier coefficient vector, and we can similarly define the inverse anisotropic fractional spectral manifold Fourier transform

$$\mathbf{f}={\mathbf{U}}_{\alpha \theta }^s\widehat{\mathbf{f}}.$$

(6)

In the next section, we will propose our anisotropic fractional spectral manifold wavelets, which have lots of attractive geometrical properties.

Wavelets functions. Given a fractional order s, an anisotropic degree $\alpha$, and a rotational angle $\theta$, we let a real-valued kernel function $g(·)$ act as a band-pass filter. Then, we give the definition of the anisotropic fractional wavelet function with scale t and located at vertex v on the mesh $\mathcal{M}$ as

$${\mathit{\psi }}_{\alpha \theta ,tv}^s(·)=\sum _{k=0}^{K-1}a\left(v\right)g\left(t{\lambda }_{\alpha \theta ,k}^s\right)u_{\alpha \theta ,k}^s\left(v\right){\mathit{u}}_{\alpha \theta ,k}^s(·).$$

(7)

In addition, in order to capture the low-frequency signal information, a scaling function at the point v is obtained by given a low-pass filter $h(·)$ in the spectral domain, namely,

$${\mathit{\xi }}_{\alpha \theta ,v}^s(·)=\sum _{k=0}^{K-1}a\left(v\right)h\left({\lambda }_{\alpha \theta ,k}^s\right)u_{\alpha \theta ,k}^s\left(v\right){\mathit{u}}_{\alpha \theta ,k}^s(·),$$

(8)

where $a\left(v\right)$ is the Voronoi area of the vertex v.

As shown in Figure 1, we give the visualization of various anisotropic wavelets centered at the same point on the shape with different parameters, including the fractional order s, the anisotropic degree $\alpha$, the rotational angle $\theta$, and the scale t.

Wavelet transform. Similar to a classical spectral graph wavelet, our Anisotropic Fractional Spectral Manifold Wavelet Transform (ASMWT) is also obtained by the inner product of signal $\mathbf{f}\in {\mathbb{R}}^{N\times 1}$ and the wavelet function ${\mathit{\psi }}_{\alpha ,\theta ,t,v}^s$ with regard to the mass matrix $\mathbf{A}$, i.e., the wavelet coefficient is given as

$$\begin{array}{cc}\hfill W_{\mathbf{f}}^s(\alpha ,\theta ,t,v)& =<\mathbf{f},{\mathit{\psi }}_{\alpha \theta ,tv}^s{>}_{\mathbf{A}}\hfill \\ \hfill & =\sum _{k=0}^{K-1}a\left(v\right)g\left(t{\lambda }_{\alpha \theta ,k}^s\right){\widehat{f}}_{\alpha \theta ,k}^s{\mathit{u}}_{\alpha \theta ,k}^s\left(v\right),\hfill \end{array}$$

(9)

Likewise, the corresponding scale function coefficient can be computed by

$$\begin{array}{cc}\hfill S_{\mathbf{f}}^s(\alpha ,\theta ,v)& =<\mathbf{f},{\mathit{\xi }}_{\alpha \theta ,v}^s{>}_{\mathbf{A}}\hfill \\ \hfill & =\sum _{k=0}^{K-1}a\left(v\right)h\left({\lambda }_{\alpha \theta ,k}^s\right){\widehat{f}}_{\alpha \theta ,k}^s{\mathit{u}}_{\alpha \theta ,k}^s\left(v\right).\hfill \end{array}$$

(10)

Note that in practical application, the parameters t and $\theta$ need to be discretized to a finite number of values. For briefness, we denote all wavelet kernel functions with discretized scales together by

$$g_{t_j}\left({\lambda }_{\alpha \theta ,k}^s\right)=\left\{\begin{array}{cc}h\left({\lambda }_{\alpha \theta ,k}^s\right)\hfill & j=0,\\ g\left(t_j{\lambda }_{\alpha \theta ,k}^s\right)\hfill & j>0.\end{array}\right.$$

This wavelet transform allows capturing multiscale and multidirectional signal information in the neighborhood of the wavelet’s located point.

4. Anisotropic Fractional Spectral Manifold Wavelet Descriptor

As discussed above, the AFSMWT can analyze signals from various diffusion directions on the shape and encode different sizes of neighborhood features. Therefore, we are permitted to build a powerful shape descriptor just utilizing the wavelet coefficients of a special signal. We choose the delta function, a very simple function, to generate our AFSMW descriptor (AFSMWD) instead of using a series of complex geometry vectors in [23,26].

For each point v on the shape $\mathcal{M}$, the Delta function centered at that point is denoted as ${\delta }_v$. To describe the multiscale local shape context around this point, we first set an appropriate value for the anisotropy degree $\alpha$. Next, we use L equally spaced rotation angles, denoted as $\theta ={\theta }_1,{\theta }_2,\dots ,{\theta }_L$, where ${\theta }_l=2(l-1)\pi /L$ for $l=1,2,\dots ,L$. To capture information from different neighboring vertices around point v, we employ a series of fractional orders ${\left\{s_i\right\}}_{i=1}^I$, where each $s_i$ is uniformly distributed within the interval $[0,1]$. Additionally, the scale parameter t is sampled into J different scales, represented as $t_1,t_2,\dots ,t_J$, to encode multiscale geometric features along each local orientation defined by ${\theta }_l$. With this setup, we compute the AFSMWT coefficients across J scales along with the corresponding scaling function coefficients. Finally, by combining all these coefficients, we define a $(J+1)$-dimensional vector as

$${\mathbf{d}}_{\alpha {\theta }_l,v}^{s_i}=\left(S_{{\delta }_v}^{s_i}(\alpha {\theta }_l,v),W_{{\delta }_v}^{s_i}(\alpha {\theta }_l,t_{1}v),\cdots ,W_{{\delta }_v}^{s_i}(\alpha {\theta }_l,t_{J}v)\right).$$

(11)

This vector effectively encodes the local shape characteristics around point v along the diffusion direction ${\theta }_l$, utilizing the AFSMW with fractional order $s_i$.

Considering the overall information from all fractional orders along this direction ${\theta }_l$, we further combine each ${\mathbf{d}}_{\alpha {\theta }_l,v}^{s_i}$ and obtain a $I(J+1)$ dimensional row vector

$${\mathbf{d}}_{\alpha {\theta }_l,v}=\left({\mathbf{d}}_{\alpha {\theta }_l,v}^{s_1},{\mathbf{d}}_{\alpha {\theta }_l,v}^{s_2},\cdots ,{\mathbf{d}}_{\alpha {\theta }_l,v}^{s_I}\right).$$

(12)

Note that in some specific applications, we can also use multiple anisotropic degrees. However, in our current work, we find using a single $\alpha$ is sufficient to construct a discriminative descriptor, as demonstrated in Section 5. Finally, we build our AFSMW for point v as follows

$${\mathbf{d}}_v=\left({\mathbf{d}}_{\alpha {\theta }_1,v},{\mathbf{d}}_{\alpha {\theta }_2,v},\cdots ,{\mathbf{d}}_{\alpha {\theta }_L,v}\right).$$

(13)

Let us now explain how to practically compute the AFSMWD. The kth fractional anisotropic Fourier coefficient of the delta function ${\delta }_v$ is given by ${\widehat{\delta }}_{\alpha \theta }^s\left(k\right)={\langle {\delta }_v,{\mathbf{u}}_{\alpha \theta ,k}^s\rangle }_{\mathcal{M}}=u_{\alpha \theta ,k}^s\left(v\right)$. Using Equations (9) and (10), we can deduce that each component of the descriptor can be expressed as

$$W_{{\delta }_v}^s(\alpha \theta ,tv)={〈{\delta }_v,{\mathit{\psi }}_{\alpha \theta ,tv}^s〉}_{\mathbf{A}}=\sum _{k\ge 0}g\left(t{\lambda }_{\alpha \theta ,k}^s\right)({{\varphi }_{\alpha \theta ,k}^s\left(v\right))}^2,$$

(14)

$$S_{{\delta }_v}^s(\alpha \theta ,v)={〈{\delta }_v,{\mathit{\xi }}_{\alpha \theta ,v}^s〉}_{\mathbf{A}}=\sum _{k\ge 0}{h\left({\lambda }_{\alpha \theta ,k}^s\right)\left({\varphi }_{\alpha \theta ,k}^s\left(v\right)\right)}^2.$$

(15)

The choice of the kernels significantly impacts the performance of the descriptor. GSP toolbox [37] provides several commonly used wavelet generating kernels such as cubic spline and Mexican hat kernel [38]. However, we still prefer to use the Meyer kernel, as it obtains many attractive characteristics. It can form a Parseval frame thus allowing the generated wavelets to hold energy conservation between the original and transformed domain, leading to easier and more accurate signal reconstruction [39]. In addition, it has a self-adaptive bandwidth, which is more suitable for analyzing signals. All of these advantages make its generated wavelet coefficients contain more useful information and thus contribute to formulating a more discriminative descriptor than other kernels. The overall computation of our AFSMWD can be found in Algorithm 1.

Algorithm 1: AFSMWD Computation

Input: The mesh $\mathcal{M}$ with N vertices, the parameters $\alpha ,I,L,K,J$ Output: AFSMWD for  $l=1:L$  do       According to [26], calculate the ALBO: ${\mathbf{L}}_{\alpha {\theta }_l}={\mathbf{A}}^{-1}{\mathbf{B}}_{\alpha {\theta }_l}$;       Solve the generalized eigenproblem:                              ${\mathbf{B}}_{\alpha {\theta }_l}{\mathit{\varphi }}_{\alpha {\theta }_l,k}={\lambda }_{\alpha {\theta }_l,k}\mathbf{A}{\mathit{\varphi }}_{\alpha {\theta }_l,k}$       to obtain the first K eigenvectors ${\left\{{\mathit{\varphi }}_{\alpha {\theta }_l,k}\right\}}_{k=1}^K$ and eigenvalues ${\left\{{\lambda }_{\alpha {\theta }_l,k}\right\}}_{k=1}^K$;       for $i=1:I$ do       Compute ${\left\{{\mathbf{u}}_{\alpha {\theta }_l,k}^{s_i}\right\}}_{k=1}^K$ and ${\left\{{\lambda }_{\alpha {\theta }_l,k}^{s_i}\right\}}_{k=1}^K$ using Equations (2) and (3);       Select the Meyer wavelet generating kernel functions $h\left({\lambda }_{\alpha {\theta }_l}^{s_i}\right)$ and $g\left({\lambda }_{\alpha {\theta }_l}^{s_i}\right)$ by utilizing the GSP-toolbox [37];       Compute $S_{{\delta }_v}^{s_i}(\alpha {\theta }_l,v)$ using Equation (15);             for $j=1:J$ do             According to Equation (14), compute $W_{{\delta }_v}^{s_i}(\alpha {\theta }_l,t_{j}v)$;             end             ${\mathbf{d}}_{\alpha {\theta }_l,v}^{s_i}=(S_{{\delta }_v}^{s_i}(\alpha {\theta }_l,v),W_{{\delta }_v}^{s_i}(\alpha {\theta }_l,t_{1}v),\cdots ,W_{{\delta }_v}^{s_i}(\alpha {\theta }_l,t_{J}v))$;       end       ${\mathbf{d}}_{\alpha {\theta }_l,v}=\left({\mathbf{d}}_{\alpha {\theta }_l,v}^{s_1},{\mathbf{d}}_{\alpha {\theta }_l,v}^{s_2},\cdots ,{\mathbf{d}}_{\alpha {\theta }_l,v}^{s_I}\right)$; end AFSMWD of point v, i.e., ${\mathbf{d}}_v=({\mathbf{d}}_{\alpha {\theta }_1,v},{\mathbf{d}}_{\alpha {\theta }_2,v},\cdots ,{\mathbf{d}}_{\alpha {\theta }_L,v})$.

5. Experimental Results

We adopt the generally used strategy to evaluate the descriptor performances including the discriminative and localized abilities and the robustness, which is measured by the quality of shape dense pointwise correspondence established via just using nearest neighbor searching in the descriptor space.

5.1. Implementation Details

Datasets. To assess the performance of the proposed descriptor, we intend to evaluate its matching results on several publicly available datasets, ranging from standard to more challenging ones. These datasets encompass a wide range of categories, including 3D human shapes as well as animal models. Some datasets introduce additional complexities, such as topological noise, which pose significant challenges for descriptors. We roughly classify them into the following benchmarks.

Near-isometric matching benchmark. This benchmark includes five datasets: FAUST [40], its remeshed version [41], SCAPE [42], its remeshed version [41], and the remeshed TOSCA [43]. The remeshed versions are more realistic but present greater challenges due to incompatible mesh structures. Both FAUST and its remeshed version feature 10 distinct human subjects, each in 10 different poses, totaling 100 models. SCAPE and its remeshed version consist of 72 poses of a single human subject, resulting in 72 models. The remeshed TOSCA dataset includes 76 models, comprising 11 cats, 9 dogs, 3 wolves, 8 horses, 6 centaurs, 4 gorillas, 1 female mannequin with 12 poses, and 2 male figures with 7 and 20 poses, respectively.

Non-isometric matching benchmark. This benchmark includes the SHREC’19 [44] and SMAL [45] datasets. SHREC’19 is composed of multiple datasets, such as FAUST, SCAPE, CAESAR [46], and SHREC14 [47], among others. Since the shapes in SHREC’19 are created using different modeling techniques and for various purposes, they exhibit diverse connectivity and poses. The presence of incomplete shapes and a large number of matching pairs further adds to the dataset’s complexity. The SMAL dataset consists of 49 four-legged animal shapes from eight species, representing families like Felidae, Canidae, Equidae, and Bovidae. The significant non-isometric deformations across species make this dataset particularly challenging for existing methods.

SHREC’16 Partiality benchmark [48]. This dataset contains nearly isometrically deformed shapes across eight classes, featuring two types of partiality: clean cuts and holes. Partial shapes often arise in real-world scenarios, where data acquisition from 3D sensors frequently leads to missing parts due to occlusions or limited views. This benchmark is crucial for evaluating the robustness of descriptors to partial missing settings.

Evaluation methodology. Further, we use correspondence quality characteristics (CQC) curves [49] and average geodesic error (AGE) to evaluate matching performance. To be specific, assuming the point x on shape $\mathcal{X}$ is matched with the point y on shape $\mathcal{Y}$, while its ground-truth correspondence point is actually $y^{*}$ on $\mathcal{Y}$, then the matching error is computed as the geodesic error $e\left(x\right)=d_{\mathcal{Y}}(y,y^{*})/\mathrm{diam}\left(\mathcal{Y}\right)$, where the geodesic distance between points y and $y^{*}$ is normalized by the diameter of shape $\mathcal{Y}$. CQC curves exhibit the percentage of matches that are at most r in geodesic error. Moreover, the AGE is obtained by $e=\left({\sum }_{i=1}^{N}e\left(x_i\right)\right)/N$; here, N is the number of vertices of shape $\mathcal{X}$. Obviously, higher CQC curves and smaller AGE express more accurate matching results.

Parameter setting. We empirically set the values of five parameters in our Algorithm 1, partly referring to previous similar approaches like ASMWD [8]. In the following experiments, unless otherwise stated, we let the number of the fractional order $I=6$, the number of the first AFLBO’s eigenvalues $K=300$, the number of the discrete scales of wavelets $J=9$, the number of rotation angles $L=9$ and the anisotropy degree $\alpha =10$.

5.2. Descriptor Evaluation and Comparisons

In this section, we will comprehensively analyze our descriptor’s performance such as discriminative ability, robustness to noises, etc., and compare our AFSMWD with several state-of-the-art spectral descriptors, including HKS [2], WKS [3], AMKS [19], AWFT [23], LPS [50], ASMWD [8], and WEDS [18].

For an overview comparison, we first summarize all the matching AGEs of all the above-mentioned datasets in

Table 1. The results of all compared descriptors on all test datasets. The numbers are the matching AGEs, and the best results are in bold. Our descriptor AFSMWD achieves the best performances in nearly all datasets, including challenging cases.

Descriptors	FAUST		SCAPE		TOSCA	SHREC’19	SMAL	SHREC’16 HOLES
	Original	Remesh	Original	Remesh	Remesh
HKS	17.71	17.61	18.86	19.24	21.01	26.49	34.40	31.89
WKS	15.14	15.14	16.04	16.44	16.83	21.65	29.79	33.69
AMKS	15.72	15.22	17.08	16.48	15.27	21.88	35.79	32.55
AWFT	9.84	13.74	14.20	14.87	13.63	27.35	24.48	24.45
LPS	16.96	17.76	18.20	18.91	18.50	27.17	34.62	27.38
ASMWD	4.99	8.64	8.18	11.65	11.77	19.00	29.29	24.42
WEDS	12.52	12.12	16.52	16.23	16.60	25.40	31.98	25.16
AFSMWD	3.89	8.42	8.03	11.55	11.17	18.71	27.88	22.31

. The results clearly show our superiority to other spectral descriptors even in challenging cases. In addition, for a more comprehensive and in-depth understanding, we successively and detailedly discuss the result of each dataset in the following with more quantitative and qualitative measures.

Evaluation of Near-Isometric Shapes. First of all, we assess the performance of descriptors on the original FAUST and SCAPE datasets. The quantitative and qualitative results are presented in Figure 2. The results consistently demonstrate that AFSMWD is more discriminative compared to other state-of-the-art descriptors. Since the shapes in the original FAUST dataset possess identical numbers of vertices and connections, which is not practical for real-world applications, we next evaluate the descriptors on the remeshed datasets, which present much greater variability in terms of shape structures and connectivity. We randomly select 100 isometric shape pairs from the remeshed FAUST, 72 from the remeshed SCAPE, and 284 from the remeshed TOSCA. All methods in the comparison utilize the publicly available codes and settings provided by the respective authors. Figure 3 illustrates the CQC curves and AGEs for each descriptor. Clearly, AFSMWD produces the most accurate matching results across all three datasets, underscoring its superior discriminative power compared to other descriptors. It is worth mentioning that as a spectral descriptor, AFSMWD naturally retains invariance to isometric deformations.

Evaluation of Non-Isometric Shapes. The task of non-isometric shape matching is much more challenging than near-isometric shape matching due to the presence of larger deformations, thus putting higher requirements on descriptor performance. We use all 430 pairs provided by the SHREC’19 to evaluate the descriptor’s discrimination ability. The AGEs can be found in the corresponding column in Table 1, and the CQC curves are demonstrated in Figure 4. Despite the challenge posed by non-isometric shapes with substantial deformations or defects, AFSMWD still delivers exceptional performance.

In addition, we use 272 shape pairs of the SMAL dataset to evaluate the performance further. Particularly, to compare the performances of our AFSMWD and the most related work ASMWD [8], we only show their two CQC curves of the SMAL dataset in the left part of Figure 5, where the superior announces again the improvement of our extended fractional wavelets to common wavelets. Note that in the right part of Figure 5, we also show the visualized Euclidean distances of the descriptors between the reference points and the left points of the shapes from the SMAL dataset as well as all points on several other shapes from the same dataset. We can observe that the bluest colors representing the shortest distances appear in the most semantically similar shape areas and are highly centralized. These characteristics verify our descriptor’s outstanding discriminative and localizable abilities.

Evaluation on Partiality Shapes. To assess how well the proposed descriptor handles shapes with missing parts, we conduct tests on the SHREC’16 Partiality dataset [48], which includes shapes with significant cuts and holes. Generally speaking, spectral descriptors struggle with partial matching because (A)LBOs and their eigensystems are highly sensitive to missing geometry [51]. However, as shown in the final column of Table 1 and the partial matching outcomes in Figure 6, our AFSMWD achieves better performance than ASMWD, highlighting the strength of AFSMWD and the improvement of the existing wavelets.

6. Conclusions

In this work, we proposed a powerful local spectral descriptor AFSMWD for shape analysis tasks. For this goal, we extended the common Anisotropic Spectral Manifold Wavelet to be more flexible to extract local shape features, which was achieved by integrating the fractional theory into the manifold wavelets. As equipped with one more parameter to control the wavelet transform involving different shape vertices, the AFSMW could reveal more hidden features than all existing signal processing tools on manifolds. By resorting to this novel tool, we construct our descriptor AFSMWD just using this novel transform coefficient of a very simple function. The proposed descriptor is verified to be localizable, discriminative, and robust, and it has achieved state-of-the-art results even in some challenging occasions. We believe that our method will be a valuable contribution to the field of computer graphics and beyond.

Limitations. Our work recently can not work very well on the SHREC’16 Topology benchmark [52]. This dataset is also challenging since the topology of each shape in several regions has been changed to create a non-intersecting manifold surface. Under this circumstance, conflicts may be generated if the wavelets have unsuitable scales and shapes to extract each point’s neighborhood features, as the incorrect structural information is encoded also. More parameters involved conversely make the problem more complicated. We believe the performance can be promoted if more in-depth research is conducted on the relations of the parameters and their settings.