1. Introduction
Finding efficient algorithms to describe, measure and compare shapes is a central problem in image processing. This problem arises in numerous disciplines that generate extensive quantitative and visual information. Among these, biology occupies a central place [
1]. In cellular biology for example, the measurements of cell morphology and dynamics by imaging techniques such as light or fluorescent microscopy yield large amounts of 2D and 3D images that need to be segmented and quantified [
2,
3,
4,
5]. Measuring shapes, computing the contribution of a shape to a potential, and more generally quantifying the effect of a vector field on a shape are also common problems for geodesists and physicists.
In a chapter titled “The Comparison of Related Forms”, Thompson explored how differences in the forms of related animals can be described by means of simple mathematical transformations [
6]. This inspired the development of several shape comparison techniques, whose aim is to define a map between two shapes that can be used to measure their similarity. An alternate and popular method is to derive features (also called shape descriptors or signatures) for each shape separately that can then be compared using standard distance functions, and those that directly attempt to map one surface onto the other, thereby providing both local and non-local elements for comparison.
One particular powerful technique for generating shape signatures is based on moment-based representations of a shape. Those representation form a class of shape recognition techniques that have been used widely for pattern recognition [
7,
8,
9]. These moments not only provide measures of the shapes, such as volume and surface areas, they also allow for the encoding of a shape with descriptors that are amenable to fast analysis. The most common of these moments are geometric:
where the integration is performed over the volume
V of the shape
S considered,
is the order of the moment, and
is a vector field over
S that may represent a potential, a grey scale for image processing, or an indicator function such that
inside the shape and 0 otherwise. These geometric moments and their invariant extensions have been used extensively in pattern recognition (for review, see for example [
10]). They are usually easy to compute, though at a high computational costs.
Spherical harmonics [
11,
12] and their rotational invariants [
13] form another class of moment-based descriptors for analyzing star-shaped objects that are topologically equivalent to a sphere. They are becoming increasingly popular in cellular bio-imaging for example, as it is usually safe to assume that cells observed through a microscope are topologically closed and equivalent to a sphere; this prior makes it possible to 0 their shape mathematically, thereby simplifying the definition of their surface and provide a shape description (see for example [
3] for representing cell organelles and more recently [
14] for studies of cell dynamics).
In many cases however, the hypothesis that the object is star-shaped does not hold. The Zernike moments, first introduced in two dimensions by Dutch Nobel price Frederic Zernike in 1934 [
15], circumvent this problem through the introduction of a radial term. 2D Zernike moments have been used in cellular biology as an alternative to Quantitative Phase Imaging [
16] for classifying breast cancer tumors based on mammograms [
17], for characterizing subcellular structures [
18], and for measuring changes in cancer cell shapes [
19]. They have proved to be superior to geometric moments in 2D image retrieval (see for example [
20]). After they were generalized to 3D by Canterakis [
21], they have been applied in many domains, such as tools for shape retrieval in computer graphics [
22,
23], terrain matching and building reconstruction [
24,
25], as well as in astronomy [
26]. Most current applications, however, are related to biology, coincident with the applications of geometric moments. In biochemistry, for example, they have been proposed as a tool for protein shape comparisons [
27,
28] and alignments [
29], to the point that they have become a standard tool [
30,
31] associated with the Protein Structure Database (PDB) [
32]. Zernike moments are also used to analyze and search the protein models [
33] generated by AlphaFold, the deep learning method that is currently used to predict the structure of proteins with high accuracy [
34,
35]. The formalism for characterizing shapes using Zernike polynomials and Zernike moments is also being used for understanding the spatial organization of nonpolar and polar residues within protein structures [
36], protein docking [
37,
38], for the analyses of protein interfaces [
39,
40,
41,
42], as well as for the identification and prediction of protein binding sites [
43]. It is noteworthy that 3D Zernike moments can capture the geometry of a shape with minimal loss. They has been used in this context to encode single-cell phase-contrast tomograms [
44]. In this paper, we are concerned with the computation of these 3D Zernike moments.
Many algorithms have been proposed for computing the 3D Zernike moments of a shape. Most of these methods, especially those used for image analysis, rely on a representation of the shape as a volumetric grid. These algorithms usually proceed in two steps, namely computing the geometric moments of the shape first, and then expressing the Zernike moments as linear combinations of those moments (see Refs. [
22,
45], and background section below). They do suffer from three major drawbacks. First, the computation of each moment has a cubic computational complexity (
, where
is the number of voxels in each dimension of the grid). Second, the volume integral in Equation (
1) is approximated by a discrete sum over the voxels, where each voxel contributes as a point-like object, usually located at its center. It is easy to see that this discretization error increases as the order of the moment increases. Finally, the relationships between geometric moments and Zernike moments lead to numerical instabilities, limiting the order
N of the moments that can be computed accurately, usually with
(see [
22,
46]).
Given a volumetric grid representation of a shape, it is possible to extract a discrete version of the surface of this shape by triangulation of an isosurface embedded in the grid (for a survey on this process, see for example Ref. [
47]). Popular methods for generating such isosurface include the marching cube algorithm [
48], the marching tetrahedra algorithm [
49] and its regularized version [
50]. Representing a shape using a discrete approximation of its surface, usually a polygonal mesh, and assuming that this shape is homogeneous (where homogeneity refers to the fact that the function
in Equation (
1) can be considered constant) is often efficient as there are exact algorithms for computing homogeneous 3D moments over such representation. An exact formula for computing geometric moments on 3D polyhedra was originally proposed by Lien and Kajiya [
51]; it has been reformulated and extended to general dimensional polyhedra several times since then (for a complete discussion of the state of the art in this field, see [
52]). Pozo et al. used those ideas and derived efficient recursive algorithms for computing geometric moments for shapes defined by a triangulation of their surfaces [
52]; those algorithms were further refined to be optimal with respect to the order of the moments [
53]. Pozo et al. then used those geometric moments to derive the 3D Zernike moments of such shapes. As mentioned above, however, the numerical instabilities associated with the relationships between geometric and Zernike moments limit the order with which those can be derived.
Recently Deng and Gwo proposed a new, stable algorithm to accurately compute Zernike moments for shapes represented as 3D grid [
54]. Their algorithm proceeds by computing these moments directly, without using the geometric moments, using recurrence relations that provide stability and efficiency. Our work presented in this paper is a counterpart to the work of Deng and Gwo. We propose a new exact algorithm for the computation of Zernike moments for shapes represented by surface meshes. Similar to Deng and Gwo, we do not use the conversion from geometric moments to Zernike moments to avoid numerical instabilities.
The paper is organized as follows. In the next section, we give some background on moments of 3D shapes, especially geometric moments and Zernike moments. We show how the former can be used to compute the latter, but explain why this may lead to numerical instabilities, especially for high order moments. The following section introduce our new, stable algorithms for computing Zernike moments of 3D shapes represented by surface-based triangular meshes. The results section introduce some experiments to validate those algorithms on synthetic data.
2. Moments from 3D Shapes
In this section, we briefly introduce the concept of moments of a shape, with three examples, the geometric moments (GM), the spherical harmonics moments (SPHM), and the Zernike moments (ZM). We show that ZM are extensions of the SPHM, and that they can be computed from the GM, but with potential problems that we describe.
We start with notations. Any point within a domain can be 0 either in terms of its vector of Cartesian coordinates with respect to an origin, or in terms of its spherical coordinates where r is the distance from to the sphere center, and and are the inclination and azimuthal angles, respectively. A shape S in the domain D is represented through its density at all points in the domain. We assume that f is square integrable over the domain D, i.e., . In the special case that the shape is homogeneous, it is represented with an indicator function with value 1 if is inside the shape, and 0 otherwise.
2.1. Moments of a Shape
The moments
of a shape are the projections of the function
f over a set of basis functions
:
where
is the complex conjugate of
. The properties of a particular moment based representation are therefore determined by the set of functions
. There are two properties of this set that are desirable, namely:
- (i)
Orthonormality. The set of functions
is orthonormal if
for all
and
in
(
is the Kronecker symbol, i.e.,
if
, and 0 otherwise).
- (ii)
Completeness. The set of functions
is complete if for all functions
:
The orthonormality property is important as it guarantees the mutual independence of the computed moments. The completeness property implies that we are able to reconstruct approximations of the original shape from its moments. Directly associated with these two properties is Parseval’s theorem. This theorem is an important property for example in Fourier analysis that states that the sum (or integral) of the square of a function is equal to the sum (or integral) of the square of its transform. It is true in fact for all complete orthonormal basis. Indeed,
2.2. Basis Function: Monomials
A very popular set of functions
are the monomials
, where
are non negative integers. The corresponding moments are referred to as geometric moments,
, and defined by
The geometric moments are easy to compute, both for grid-based and surface-based representations of shapes. The geometric monomials, however, are neither orthonormal, nor complete.
2.3. Basis Function: Laplace Spherical Harmonics
If the domain is the sphere
, a point
is characterized by its inclination angle
and azimuthal angle
only. A function
f on
can then be represented through its spherical harmonics. They form a Fourier basis on a sphere much like the familiar sines and cosines do on a line. The spherical harmonic
is defined by
where
is a normalization factor:
and
are the associated Legendre polynomial.
is defined for
l non negative integer and
m integer, such that
. It is enough, however, to compute the spherical harmonics for
, as we have the relationship,
We note the definition of the harmonic polynomial,
where
is the point with spherical coordinates
. The spherical harmonics are orthonormal:
The spherical harmonics moments
are defined as:
Note that these moments are complex, as the spherical harmonics are complex.
Just like Fourier series are complete, the spherical harmonics are complete on the sphere. It is important to notice, however, that they cannot be computed on a volumetric shape without some modifications, such as the use of solid harmonics [
55], or with the introduction of Zernike polynomials, which are presented in the following subsection.
2.4. Basis Function: The Zernike Polynomials
The spherical harmonics are defined on the sphere. Zernike [
15] in 2D and later Canterakis [
21] in 3D have shown that they can be expanded to account for the whole ball by using the Zernike polynomials. Let
be a point inside the unit ball
with spherical coordinates
. The value of Zernike polynomial
at
is given by:
where
are the spherical harmonics define above, and
are polynomials in the radial coordinate
r:
where
The non negative integer
n is the order the of Zernike polynomial,
l is an integer that is restricted so that
and
be an even number,
, and
. The coefficients in
were chosen to guarantee that the Zernike polynomials are orthonormal, a property expressed in the following equation:
The Zernike moments
of a shape
S whose density inside the unit ball is defined by the square integrable function
f are then defined as
Note that
m can be positive or negative, as it belongs to
. It is enough, however, to compute the moments for
m non negative as we have (see for example [
22]):
Reconstruction. The Zernike polynomial form a complete basis of
, we hence have
where
l is an integer that is restricted so that
and
be an even number. In practice, we can use a finite number of terms to approximate
f.
2.5. Computing the Zernike Polynomials from the Geometric Moments
Although the Zernike polynomials are usually defined with respect to spherical coordinates, they are actually polynomial functions on Cartesian coordinates
. To show this, let us first rewrite the Zernike polynomials
where
and
is the harmonic polynomial (see Equation (
5)) and
. The harmonic polynomial can be expressed in Cartesian coordinates [
22], but it is enough to know that it is a polynomial of total degree
l. We can thus write
with
r,
s and
t non negative integers and
some known complex numbers. We now have
with
Finally, after replacing the Cartesian expression for
(Equation (
17)) into Equation (
12), we get an expression of the Zernike moments
as a function of the geometric moments
G:
This allow for a simple algorithm for computing the Zernike moments of a shape from its geometric moments. Similar algorithms have been proposed [
22,
52] for the same task. Such an algorithm is theoretically very efficient, once the geometric moments have been computed, as it is independent of the size of the mesh representing the shape or the number of grid point in a voxel representation of the shape. There are, however, numerical issues with this formula that we discuss below.
2.6. Numerical Instabilities Associated with the Zernike Polynomials
Let us first look at the radial polynomials.
is a polynomial of degree
n. For large values of
n, special care is needed for computing them, and direct application of Equation (
9) is bound to numerical instabilities, as described in
Figure 1.
The values of
as a function of
r, based on a stable evaluation of the polynomial function vary in the interval
, where the largest value is reached for
. Numerical application of Equation (
9) wrongly indicates that
varies in the interval
(results not shown). This is due to the fact that the coefficients
are large (up to
), as illustrated in panel B of
Figure 1. Those coefficients alternate from positive to negative due to the presence of the term
, leading to large cancellations and ultimately to a small value for the polynomial. Computing correctly those cancellations requires very high precision usually not available with standard double precision in programming languages. It is possible to use arbitrary precision libraries to solve this issue, but it is in fact not necessary. As was noticed multiple times for the 3D Zernike radial polynomials (see for example [
56,
57]) the radial polynomial
can be expressed as a Jacobi polynomial:
Equation (
19) allows the results available in the literature for the Jacobi polynomials to be translated for the 3D radial functions. In particular, we have the following recurrence relation (it was also derived in [
54])
where the coefficients
are defined as:
This recursive formula is only valid for
. This can be addressed by noticing that
This recurrence allows for a stable computation of all , even for large orders.
The geometric moments of a shape can be computed accurately even for large orders, see for example [
52,
53]. Those moments can then be used to evaluate the 3D Zernike moments, as described in the section above. Converting the geometric moments to Zernike moments require, however, that the factor
be computed (where
refers to the indices for the Zernike moments, while
refers to the indices for the geometric moments). As defined in (
18), the factors
depend on the coefficients
and therefore they are bound to suffer from the same numerical instabilities. We illustrate this in
Figure 2.
As expected, the
vary significantly over a large range of values. This was already observed by Berjón and colleagues in their attempts to parallelize the computation of Zernike moments [
46] and is the main reason that the computation of Zernike moments is usually limited to order below
. One solution to solve this problem would be to derive recurrence relationships for the
. Pozo et al. provided such a recurrence. However, their relationships still involve the computations of the factors
(see their Equation (13e)) and as such do not solve the numerical instabilities.
Deng and Gwo [
54] proposed a different approach in their attempt to compute Zernike moments for a shape described on a grid, which is to bypass the computation of the geometric moments. In the following, we propose a similar approach for computing Zernike moments for a shape described by a surface-based triangular mesh.