Motion Artifact Detection Based on Regional–Temporal Graph Attention Network from Head Computed Tomography Images

Abstract

Artifacts are the main cause of degradation in CT image quality and diagnostic accuracy. Because of the complex texture of CT images, it is a challenging task to automatically detect artifacts from limited image samples. Recently, graph convolutional networks (GCNs) have achieved great success and shown promising results in medical imaging due to their powerful learning ability. However, GCNs do not take the attention mechanism into consideration. To overcome their limitations, we propose a novel Regional–Temporal Graph Attention Network for motion artifact detection from computed tomography images (RT-GAT). In this paper, head CT images are viewed as a heterogeneous graph by taking regional and temporal information into consideration, and the graph attention network is utilized to extract the features of the constructed graph. Then, the feature vector is input into the classifier to detect the motion artifacts. The experimental results demonstrate that our proposed RT-GAT method outperforms the state-of-the-art methods on a real-world CT dataset.

1. Introduction

Regular physical examination can help to improve the health of hospital residents. With the development of artificial intelligence technology, the construction of intelligent medical systems, the alleviation of medical resource shortages using computer-aided methods, and even the replacement of doctors to screen patients have become the main research directions. The work in this paper belongs to the core field of “virtual doctor” research in smart cities, which can promote future intelligent medical treatment.

Image quality detection technology based on computer vision is the core of the intelligent medical field. Medical imaging technology plays an important role and has provided clinicians with an effective tool for diagnosis. In particular, computed tomography (CT) has emerged as an important tool and has been widely used due to its low cost, high efficiency, readily obtainable results, and non-invasive nature [1]; it contains valuable information. However, due to an individual’s motion, motion artifacts often obscure the image and threaten the accuracy of judgement in clinical practice. Thus, in academia and industry, the detection of motion artifacts has become an important area in CT quality analysis and has attracted widespread attention [2,3]. Great efforts have been made to explore the internal mechanism of motion artifact detection throughout the years. Conventional methods manually classify and label each image according to the attributes, which is tedious and time-consuming. Moreover, evaluating the quality of CT images is a challenging task that requires extensive clinical experience and is error-prone.

Researchers have carried out a lot of valuable work to automatically assess CT image quality [4,5,6,7,8,9]. Several studies have mainly focused on handcrafted feature designation. Deep learning methods [10,11,12,13,14], such as convolution neural networks (CNNs), have achieved unprecedented performance in the processing of Euclidean data [15,16,17]. However, there are some limitations and challenges when it comes to the medical domain. For instance, these methods neglect useful relationships between images and are still under development for graph-structured data. Related nodes, i.e., the neighbors of a node, can give insights to better understand the current image. The relationship between quality and topological properties may be uncovered through combining domain knowledge at the macro and micro levels; this would help in improving the accuracy and in understanding the formation mechanism of motion artifacts. Because of the black box nature of deep learning methods, features are extracted implicitly, and there are few clues as to how decisions are made; this makes it difficult to understand why the algorithms predict a certain label.

Complex networks are highly simplified models of complex systems [18,19,20] and provide a way to understand the nature and function of complex systems. They have attracted increasing attention because they are powerful, flexible, and universal. For example, the presentation of problems in and topological characteristics of complex networks have been studied [21,22]. Many interdisciplinary applications, including social networks [23,24], software networks [25], biology [26,27,28,29], and earthquake networks [30], have emerged based on complex networks. The topological characteristics of the network can be obtained, and the network can be analyzed. There are few studies on image representation using complex networks. There is, therefore, still a lot of room for research on the application of complex networks to images.

To tackle the aforementioned problems, graph-based analysis methods have been established to preserve the structure and extract node features. Deep learning technologies are being used to process graph-structured data and learn the representation of networks, such as graph neural networks. Based on the feature learning ability of GNNs, they can realize effective motion artifact detection under a limited number of samples, meeting the needs of clinical practice [31]. Recently, graph convolutional networks (GCNs) [32,33,34] have been demonstrated to be effective in dealing with graph data for image classification tasks. An end-to-end GCN, ImageGCN [35], which learns the representation of an image, has been applied to chest X-ray images for disease identification. When it comes to medical domain knowledge, many researchers are devoted to analyzing the critical role of the shape and location. X-ray images of the cervical spine have been automatically segmented using the shape-aware deep segmentation network UNet-S. ‘S’ refers to the use of the updated shape-aware loss function [36]. Researchers have also verified that it is critical to take rib region information into account and confirmed the necessity of region-specific features in the segmentation task [37], which may benefit the classification task. It has also been demonstrated that the incorporation of prior anatomical knowledge may effectively improve the performance. In [38], it has been demonstrated that the PPGs used to extract RR are not very clean, and this is an important reason for which they are destroyed by motion artifacts. This poses a challenge to the reliable extraction of these metrics. A graph attention network (GAiA-Net) for image denoising is proposed to deal with the increase in noise complexity in noisy images and solve the problem whereby existing artifact detection methods cannot provide satisfactory solutions. Because the presence of motion artifacts (Mas) hinders the accurate analysis of EDA signals, the study in [39] presents a machine learning framework for automatic motion artifact detection on electrodermal activity signals.

The motivations of this study are the core issues in medical image classification identified from the above analysis, i.e., how to model the relationships between head CT images and efficiently fuse domain knowledge to improve the classification accuracy. To achieve these goals, in this work, it is essential to take region heterogeneity into account. First, we introduce complex network theory to model the relationships between head CT images based on anatomical prior knowledge, called the RT-graph. Efforts have been dedicated to promoting a systematic understanding of the relationship between topology, function [40,41,42,43,44,45,46], and network characteristics. Identifying this relationship may help in understanding images of different quality, leading to better explainability. Second, to learn the relationships between different images, we design a medical image classification algorithm based on GCN, which takes the degree into consideration, to learn the RT-graph. Compared with the popular CNN-based image classification method, the proposed RT-GAT approach can model graph-structured data to learn informative representations for nodes. Intuitively, by examining the content of each slice and comprehensively analyzing the relationships between slices, motion artifacts can be detected; from these, clinical diagnoses, prognoses, treatment plans, and other medical decisions may be better formulated.

The main contributions in this work are summarized as follows.

A Regional–Temporal Graph Attention Network is developed for motion artifact detection from head CT images. From the perspective of levels, the quality assessment problem is formulated as a node classification issue at the slice level, with node attributes acquired at the pixel level.

Network information, including the node feature information and the structure information extracted from the CT network, is utilized to detect motion artifacts via a two-stage decision scheme.

An improved GNN model called RT-GAT is presented and embedded into the execution of the proposed method, such that the features of CT image quality can be highlighted using the designed community detection and anatomical prior knowledge.

2. Materials and Methods

2.1. Basic Notations and Problem Statement

Artifacts may cause a sharp decline in the quality of the image, compromising post-processing and greatly affecting the accuracy of a diagnosis; they bring serious confusion to the diagnostic process and lead to deviations in diagnosis [47,48].

In this investigation, for simplicity, there are some assumptions.

The quality of CT images is degraded merely by motion artifacts and is evaluated based on whether motion artifacts have affected the diagnosis process.

A single slice contains several pixels. All the slices are of the same resolution of $512\times 512$.

The qualities and anatomical prior knowledge (APK), i.e., region information, of a slice are correctly labeled.

2.2. Pipeline of the Proposed RT-GAT Method

The schematic diagram of the proposed RT-GAT method for motion artifact detection is illustrated in Figure 1. RT-GAT’s architecture consists of four key modules: (1) graph construction; (2) feature extraction; (3) training and learning; and (4) classification and motion artifact detection.

First, CT images are converted into an RT-graph. The feature vector is input into the classifier to detect the motion artifacts.

The framework of the proposed motion artifact detection method based on the Regional–Temporal Graph Attention Network (RT-GAT) is illustrated in Figure 1.

2.2.1. Graph Construction

Based on the RT-graph model, using complex network theory, CT images are expressed in a macro manner. Suppose that the CT images can be modeled as an undirected, unweighted graph at the region level. The graph can be denoted as G = {V, E, A}. V represents the node set of the graph with the number of |V| = N, where N is the number of total samples, and E is the edge set, where the slices representing nodes are connected in the form of a fully connected undirected graph. At the region level, a node corresponds to a slice, and an edge represents the relation between the two slices based on the labeled anatomical prior knowledge (APK) and region information. An edge is generated if two slices belong to the same region, with the same quality between different individuals, or the slices belong to the same individual, as seen in Figure 2, where the solid circle indicates individuals and the dashed circle indicates slices. At the pixel level, the attributes of the graph are obtained based on the complex network [49].

Specifically, Figure 2 shows an example of two types of nodes and two types of edge connections. To consider the relationships between CT images, an adjacency matrix was formed using complex network theory. A$\in {\mathrm{R}}^{\mathrm{m}\times \mathrm{m}}$ is the adjacent matrix demonstrating the connections between any two nodes; it is a symmetric connectivity matrix and a representative description of the graph structure in matrix form. The value of the element in the adjacent matrix is 0 or 1. The definition of the adjacent matrix is shown in Equation (1):

$$\mathrm{ai},\mathrm{j}=\left\{\begin{array}{c}0,while V_i,V_j are not connected\\ 1,while V_i,V_j are connected\end{array}\right.$$

(1)

As the characterization and identification of CT images require a methodology that can express the context surrounding each slice, both the local and global CT image characteristics of the network were joined, and multi-level exploration was used to better characterize the CT image. Thus, we also took pixel-level graph construction into consideration. An image was represented as a graph, with each pixel denoted as a node, and an edge was generated according to the location and intensity between two pixels. Pixel-level graph construction was conducted in a micro manner, and topological features, such as the average degree, average clustering coefficient, and average path length, were extracted as they could be used to explain the internal mechanism.

The network topological characteristics [19] are explained in detail as follows.

Degree [18]: In an undirected graph, the degree of a given node i is the number of nodes directly connected to it. The degree is the most fundamental characteristic and measure of a node.

Clustering coefficient [20]: The clustering coefficient implies the degree of clustering of the network.

$$C_i=\frac{2E_i}{k_i(k_i-1)}$$

(2)

A gray-level co-occurrence matrix (GLCM) reflects the spatial correlation characteristics of gray values and can effectively describe the spatial relationships between pixels in two-dimensional CT images. In order to quantitatively describe the artifacts in CT images, the gray-level co-occurrence matrix (GLCM) based on statistical theory is used to analyze the CT scan images of the head and further explore the details of microstructure development. In this study, we computed GLCM-based features. In 1973, Haralick and Shanmugam 35 proposed GLCM features based on texture attributes. We extracted the GLCM texture features from the CT images of the head and applied them to the texture analysis. We calculated various GLCM features, such as contrast, dissimilarity, homogeneity, energy, and correlation.

$$P_{\Delta x,\Delta y}(i,j)={\displaystyle \sum _{x=1}^N{\displaystyle \sum _{y=1}^N\left\{\begin{array}{c}1\hspace{1em}if\hspace{1em}I(x,y)=i,\hspace{1em}\mathit{and}\hspace{1em}I(x+\Delta \mathrm{x},\mathrm{y}+\Delta \mathrm{y})=j\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathit{otherwise}\hfill \end{array}\right.}}$$

(3)

where i,j indicate the gray-scale pixel values, and P(i,j) is the number times that the pixel pairs (i,j) are observed in the input image.

Contrast: This feature measures the local gray correlation in the image. The greater the difference in gray values in the image, the sharper the edge of the image and the greater the contrast.

$$Contrast={\displaystyle \sum _{i=0}^{N_g-1}{\displaystyle \sum _{j=0}^{N_g-1}{p}_{i,j}{(i-j)}^2}}$$

(4)

Dissimilarity: This feature reflects the local contrast change in the image. When the local contrast increases, the difference increases, but when the local contrast decreases, the difference decreases.

$$Dissimilarity={\displaystyle \sum _{i=0}^{N_g-1}{\displaystyle \sum _{j=0}^{N_g-1}{p}_{i,j}\left|i-j\right|}}$$

(5)

Homogeneity: A higher value indicates a smaller difference in gray-scale tone between two objects, while a low value indicates a greater change in contrast.

$$Homogeneity={\displaystyle \sum _{i=0}^{N_g-1}{\displaystyle \sum _{j=0}^{N_g-1}\frac{P_{i,j}}{{(i-j)}^2}}}$$

(6)

Energy: Energy is used to calculate the number of pixel pair repetitions and the amount of randomness in the image texture. This feature reflects the uniformity of the gray distribution in an image. The more concentrated the gray distribution of the image, the greater the energy. The more dispersed the gray distribution of the image, the smaller the energy. When the pixels are closely related, the energy value increases significantly.

$$Energy={\displaystyle \sum _{i=0}^{N_g-1}{\displaystyle \sum _{j=0}^{N_g-1}{(p_{i,j})}^2}}$$

(7)

Correlation: This feature indicates the relationship between the pixel and those around it. Its value can be between −1 and 1 depending on whether the image has a negative or positive correlation. It reflects the local gray-level correlation of the image and measures the similarity of the elements of the gray-level co-occurrence matrix in the direction of rows or columns. The element values of the gray-level co-occurrence matrix have smaller differences and greater correlation. The greater the difference in the element values of the gray-level co-occurrence matrix, the smaller the correlation.

$$Correlation={\displaystyle \sum _{i=0}^{N_g-1}{\displaystyle \sum _{j=0}^{N_g-1}{p}_{i,j}\frac{(i-\mu )(j-\mu )}{{\sigma }^2}}}$$

(8)

2.2.2. Graph Attention Network

The number of nodes in different regions may differ, as may the degree of the node. Thus, here, we take the graph convolution network into consideration. For an image, original features and topological features were considered in a comprehensive way as node attributes. In this way, the physical and topological properties of the head CT images were investigated as attributes of the corresponding node at the macro level, as seen at the micro level.

In this paper, the graph attention network (GAT) is used to model the whole graph structure in the feature extraction part of graph nodes to obtain the corresponding feature expression of CT image nodes in the network. The GAT model is improved based on the GCN. The attention mechanism can assign different weights to different nodes, and it depends on pairs of adjacent nodes instead of a specific network structure in training.

Assuming that there are N nodes in the graph; the node vector input into the network is recorded as follows:

$h=\left\{h_1,h_2,\dots ,h_N\right\}$, $h_i\in R^F$. The output vector of the graph attention network is recorded as follows: $h^{\prime }=\left\{h_1^{\prime },h_2^{\prime },\dots h_N^{\prime }\right\}$,$h_i^{\prime }\in R^{F\prime }$. The model used in this paper introduces a self-attention mechanism to calculate the information aggregation between nodes, and the corresponding formula is as follows:

$$e_{ij}=a(W{h}_i,W{h}_j)$$

(9)

where $e_{ij}$ represents the importance of node $j$ to node $i$.

$${\alpha }_{ij}={\mathrm{softmax}}_j(e_{ij})$$

(10)

The attention mechanism used in this paper is implemented by a single-layer feedforward neural network, in which the activation function is the LeakyReLU function, and the formula for the calculation of the attention score can be extended to Equation (5).

$${\alpha }_{ij}=\frac{\exp \left(\mathrm{LeakyReLu}\right({\beta }^{\mathrm{T}}\left[W{h}_i\left|W{h}_j\right.\right]\left)\right)}{{\displaystyle {\sum }_{k\in N_i}\exp \left(\mathrm{LeakyRelu}\right({\beta }^{\mathrm{T}}\left[W{h}_i\left|W{h}_j\right.\right]\left)\right)}}$$

(11)

wherein ${\beta }^{\mathrm{T}}$ is a trainable parameter of the feedforward neural network, and $W$ is another trainable parameter. The multi-head attention mechanism is added to the GAT. The eigenvector calculated by the K-head attention mechanism is used to concatenate, and the corresponding output eigenvector is expressed as follows:

$$h_i^{\prime }={\displaystyle \prod _{k=1}^K\sigma ({\displaystyle \sum _{j\in N_i}{\alpha }_{ij}^k{W}^k{h}_j})}$$

(12)

$W^k$ is a parameter matrix that linearly transforms the input vector. For the output of the last layer, it is found that the performance is not good when concatenating the vectors, so the average method is used to calculate the feature vector to predict the last layer.

$$h_i^{\prime }=\sigma (\frac{1}{K}{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{j\in N_i}{\alpha }_{ij}^k{W}^k{h}_j}})$$

(13)

$h_i^{\prime }$ is the feature vector that is extracted via the graph attention network, which needs to be input to the output layer.

2.2.3. Motion Artifact Detection Evaluation

We used machine learning methods and deep learning methods for classification. A deep learning method, RT-GAT, was used for feature extraction after graph construction. We also compared our framework to the GCN, which does not take attention into consideration, and a CNN (backbone: ResNet-50), a deep learning method for Euclidean data.

In the terms of medical image quality assessment, the labels may represent the quality of the medical image. For the binary classification problem, CT images with and without artifacts are determined to be positive and negative, respectively. See in

Table 1. Metrics of motion artifact detection judgement.

		Predicted
		1	0
True	1	True Positive (TP)	False Negative (FN)
False	0	False Positive (FP)	True Negative (TN)

, True-positive (TP) cases were those in which the artifact-affected image was recognized as artifact-affected, whereas false-positive (FP) cases were those in which an artifact-free image was recognized as artifact-affected. Furthermore, true-negative (TN) cases were those where artifact-free images were recognized as artifact-free, whereas false-negative (FN) cases were those where artifact-affected images were incorrectly recognized as artifact-free.

The accuracy was determined using Equation (3) for performance evaluation.

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{TP+TN}{TP+FP+FN+TN}$$

(14)

$$\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{TN}{FP+TN}$$

(15)

$$\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{TP}{TP+FN}$$

(16)

3. Results

3.1. Dataset

For CT images, we used the Digital Imaging and Communications in Medicine (DICOM) format. To test the performance of the proposed method, a real-world dataset consisting of 480 samples collected from the Chinese Neusoft Medical CT scanner (Chongqing, China) was used. The ground truth of all training and testing datasets was assessed by a neuroradiologist with 10 years’ experience in interpreting and analyzing CT. The slices were categorized into two classes: artifact-affected images and artifact-free images. Typical images from the different sections are presented in Figure 3. The one with artifacts (a) shows fissures; however, the artifact-free image shows no fissures.

After RT-graph construction, we obtained a corresponding graph (see Figure 4).

As shown in Figure 4, the graph was constructed based on regional–temporal information. Multi-dimension network topological characteristics combined with physical characteristics were fused as node feature vectors. From the level perspective, at the pixel level, topological characteristics were obtained as node attributes, and at the slice level, an RT-graph was constructed. As seen from Figure 5a–h, community detection was conducted, and the communities from 1 to 8 are demonstrated to show the clustering of the CT images.

3.2. Performance Metrics

We evaluated the model’s performance on the test set in terms of accuracy, sensitivity, and specificity. As shown in

Table 2. Performance of different methods.

	Sensitivity	Accuracy	Specificity
CNN	85.67%	78.67%	71.67%
GLCM + RF	87.00%	88.00%	89.00%
GLCM + SVM	86.00%	88.00%	93.00%
GCN	89.1%	90.3%	88.5%
GAT	89.5%	90.8%	87.2%
RT-GAT	90.7%	92.3%	89.1%

, the proposed method works well in clinical practice.

As shown in

Table 3. Information utilized in different models.

	Region-Level Global Information	Pixel-Level Local Information	Topological Features	Regional–Temporal Information	Attention Mechanism
CNN	×	√	×	×	×
GLCM + SVM	×	√	×	×	×
GLCM + RF	×	√	×	×	×
GCN	√	√	√	×	×
GAT	√	√	√	×	√
RT-GAT	√	√	√	√	√

, we evaluated the performance of RT-GAT in terms of accuracy, sensitivity, and specificity. The classification performance of the RT-GAT approach was evaluated in comparison to the CNN approach, without taking anatomical prior knowledge into consideration, as well as the GCN approach, GAT, and machine learning methods.

The proposed RT-GAT method has been compared with the DL-based baseline approach, a CNN, which implies that a deep learning method that does not take regional characteristics into consideration may not be sufficient for motion artifact detection. It has been demonstrated that the topology-aware method outperforms CNN-based pixel-wise classifiers, where the training images do not incorporate the topological information.

It can be observed from Table 2 that the proposed method, RT-GAT, achieved the best classification performance, followed by GAT, GCN, GLCM + RF, GLCM + SVM, and CNN. This could be attributed to the regional–temporal construction method and attention mechanism, which improved the classification performance.The information of the topological level is shown in Table 3.× means not include and √ means include.

4. Discussion

In this paper, a novel robust strategy that incorporates regional–temporal information with a graph attention network, called RT-GAT, has been proposed for the detection of motion artifacts. The main outcome of this work is that RT-GAT has achieved impressive performance when acting as a feature extractor in a classification task on graph-structured data. By formulating a motion artifact detection problem as a node classification problem, a comprehensive description of head CT images has been developed to effectively express multi-level characteristics. The results show that the topology-aware method with topological information outperforms pixel-wise classifiers.

In addition, an interpretable multi-level quality assessment method has been carried out on head CT images, which helps to open the “black box”’ associated with these methods. Considering that regional information is essential and helpful in classification tasks and leads to better explainability, we used both regional and temporal information to recognize the head CT images in a more comprehensive way. Moreover, related nodes, i.e., the neighbors of a node, can give insights to better understand the current image. With a combination of domain knowledge at the macro and micro levels, the relationship between quality and topological properties has been uncovered, which helps to improve the accuracy and explains the formation mechanism of motion artifacts.

The experimental results obtained on real clinical data have demonstrated that the RT-GAT method can classify the quality of head CT images effectively, even with a limited number of labeled samples, which better fits the clinical scenario. It has been demonstrated to be robust and effective in characterizing CT images of different quality.

In this work, the ground truth of all training and testing datasets was obtained by a neuroradiologist with 10 years’ experience in interpreting and analyzing CT. However, it is necessary to train and test other datasets in the future.

The proposed RT-GAT method introduces a new line of research and may serve as a valuable tool in assessing the quality of CT images. In clinical practice, motion artifacts exist in different medical image modalities, such as magnetic resonance imaging (MRI), ultrasound, and positron emission tomography (PET). The generality and feasibility of the proposed strategy need to be verified on multi-modal medical data before it can ultimately benefit the quality assessment research and medical imaging community.