Edge-Group Sparse Principal Component Thermography for Defect Detection in an Ancient Marquetry Sample ^†

Abstract

Nondestructive inspection (NDI) has immensely contributed to the restoration of historic and artistic works. As one of the most common used NDI methods, active thermography is an easy-to-operate and efficient technique. Principal component thermography (PCT) has been widely used to deal with thermographic data for enhancing the visibility of subsurface defects. Unlike PCT, edge-group sparse PCT introduced herein enforces sparsity of principal component (PC) loadings by considering the spatial connectivity of thermographic image pixels. The feasibility and effectiveness of this method is illustrated by the experimental results of the defect characterization in an ancient marquetry sample with a fir wood support.

1. Introduction

In the restoration of historic and artistic works, nondestructive inspection (NDI) can help us with a good understanding of the state of antiques or finding the invisible details. As one of the commonly used NDI methods, active thermography is an imaging procedure [1], which is based on the analysis of heat flow induced by an energetic excitation of a test object. The resulting heat flow is influenced by interior material layers and defects. As a result, the inhomogeneity caused by defects can be captured on the object surface by infrared cameras.

Active thermography has the characteristics of detection in large areas, recording in real time, and being easy to operate. Despite these advantages, objects can become indistinguishable in the thermal images because of the existence of inhomogeneous backgrounds. To solve this problem, plenty of thermographic data analysis methods have been developed. Among them, principal component thermography (PCT) [2] is widely recommended.

PCT can capture the linear relationship between pixels and extracts the latent patterns of thermographic data with loading images. However, the non-sparsity of PCT loadings (called empirical orthogonal functions in the original paper [2]) often leads to results that are difficult to interpret. Motivated by the fact that defects always behave as spatially connected pixel groups in the thermal images, an edge-group sparse PCT (ESPCT) method is utilized for thermographic data analysis to reveal the defects in an ancient marquetry sample. Inspired by the research in bioinformatics [3], ESPCT considers the spatial connectivity among pixels and imposes sparsity on the loadings. By doing this, the defects can be revealed more clearly.

2. Methodology

As shown in Figure 1, the thermographic data set recorded in an experiment can be viewed as a three-dimensional matrix with a size of $x\times y\times n$, where x $\times$ y represents the number of pixels contained in each thermal image and n denotes the number of frames collected during the process of heating and cooling. Before conducting ESPCT, each frame is unfolded to a column vector. Without loss of generality, it is assumed that the unfolded thermographic data matrix has been centered.

Figure 1. Structure of thermographic data.

The optimization problem of extracting the first principal component (PC) is formulated as:

$$\underset{{‖p‖}_2\le 1,{‖v‖}_2\le 1}{\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}} p^{T}Xv {s.t. ‖p‖}_{ES}\le k,$$

(1)

$${\Vert p\Vert }_{ES}=\underset{\forall {\mathcal{G}}^{\mathrm{’}}\subseteq \mathcal{G},\mathit{support}\left(p\right)\subseteq V\left(\mathcal{G’}\right)}{\min }\left|{\mathcal{G}}^{\mathcal{’}}\right|,$$

(2)

where $p$ is the first loading vector of dimension $m \times$ 1, $\mathrm{v}=X^{T}p$ is the first PC of dimension $n \times$ 1, and ${‖\cdot ‖}_2$ denotes the $L_2$ norm. $\mathcal{G}$ represents a group structure, which is an edge set denoting the connectivity among pixels. For thermographic data analysis, each pixel connects only to its eight nearest neighbors. In Fig. 1, the nearest neighbors of the pixel in blue are illustrated as the orange points. ${\mathcal{G}}^{\mathcal{’}}$ is a subset of $\mathcal{G}$. $V\left({\mathcal{G}}^{\mathcal{’}}\right)$ is a vertex (pixel) set induced from the edge set ${\mathcal{G}}^{\mathcal{’}}$. $\left|{\mathcal{G}}^{\mathcal{’}}\right|$ denotes the number of elements of ${\mathcal{G}}^{\mathcal{’}}$. Support($p$) denotes the set of indices of the nonzero elements in $p$. ${\Vert p\Vert }_{ES}$ is an edge-group sparse penalty (ES-penalty) leading to a sparse loading vector whose nonzero elements are determined based on some important edges in $\mathcal{G}$, where the importance of each edge is quantified with the weight ($w_{i,j}$) on it. Here, i and j are the indices of pixels, $i,j\in [1,m]$.

$$w_{i,j}=\sqrt{{z_i}^2+{z_j}^2}$$

(3)

where $z_i$ is the i-th element in the vector $z=Xv$. $k$ is an adjusting parameter that controls the number of selected edges, which in turn controls the sparsity of the loading vector. The above-mentioned optimization problem can be solved by applying a greedy strategy until convergence. After obtaining the first PC and loading pair, the following components can be calculated sequentially. More details of the learning algorithms can be found in the literature [3].

To visualize the defects, each loading vector can be reshaped to a two-dimensional matrix of size $x\times y$. Accordingly, loading images are produced for visual analysis.

3. Ancient Marquetry Sample

This ancient sample (Figure 2) is composed of three parts. The deepest layer is the support made of fir wood, in the middle part there is the animal glue, while the upper layer is the decorative layer made with multiple materials such as pearl (white tesserae), a horn of bovine horn (some brown/black tesserae), boxwood, among others. In the left subplot of Figure 2, seven defects are labeled.

Figure 2. (top) ancient marquetry sample, (middle) back of the sample, and (bottom) raw thermal images collected at the 0.03, 0.67, 1.67, and 2.33 s during the heating and cooling process.

4. Experimental Results

The authors applied ESPCT to the thermographic data of the ancient marquetry and compared the result with PCT. The experimental setup is shown in Figure 3. The sample was heated by two flashes (FORCE) which delivered a thermal pulse of energy 2560 J within 3 ms. The thermal images were photographed with an infrared camera (TAS-G100EXD, NEC) whose resolution is 320 $\times$ 240 pixels and sampling rate is 30 frames per second.

Figure 3. Experimental setup.

The region of interest is 226$\times$241 pixels. As shown in Figure 2, identifying the defects from the raw thermal images is difficult because of the effect of inhomogeneous background. After unfolding, the size of the thermographic data matrix $X$ is 54,466 $\times$ 70. The analysis results, that is, the loading images, of both PCT and ESPCT are shown in Figure 4. In the ESPCT analysis, k was selected to be 2800, which was specified according to the estimated size of the defective regions. PCT discovers the missing tesserae on the left (A, B, C, D, and E), whereas the other defects (F and G), i.e., the detachments, are not clearly detected because of their depths. The loading image corresponding to PC2 extracts most of the defect features, whereas other loadings also contain some relevant information. Such information dispersion is not desired. Meanwhile, the background patterns retained in the loading images are also unfavorable for defect identification. ESPCT provides much clearer results, in that all defects of missing white tesserae and the detachment F are well detected by the loading corresponding to PC1, while the detachment G is discovered by PC3. Other loading images, such as PC2 and PC4, contain only noises. Moreover, according to the signal-to-noise ratio (SNR) [4] presented in Table 1, ESPCT demonstrates much a better SNR for every defect.

Table 1. SNR values of PCT and ESPCT.

Method	A	B	C	D	E	F	G*
PCT	5.52	4.46	3.18	4.19	5.06	2.05	4.32
ESPCT	18.68	13.75	3.20	10.14	16.12	6.11	12.97

Figure 4. PCT (upper) and ESPCT (lower) loading images corresponding to PC1-PC4.

5. Conclusions

In this study, a novel thermographic data analysis method, named ESPCT, is adopted for defect detection in an ancient marquetry sample. Not only the missing tesserae but also the invisible detachments are successfully identified. Compared with PCT, ESPCT shows better performance in terms of feature extraction by incorporating spatial connection information and sparsity.