An Unsupervised Approach for Treatment Effectiveness Monitoring Using Curvature Learning

Abstract

Contrast-enhanced ultrasound could assess whether cancer chemotherapeutic agents work in days, rather than waiting 2–3 months, as is typical using the Response Evaluation Criteria in Solid Tumors (RECIST), therefore avoiding toxic side effects and expensive, ineffective therapy. A total of 40 mice were implanted with human colon cancer cells: treatment-sensitive mice in control (n = 10, receiving saline) and treated (n = 10, receiving bevacizumab) groups and treatment-resistant mice in control (n = 10) and treated (n = 10) groups. Each mouse was imaged using 3D dynamic contrast-enhanced ultrasound with Definity microbubbles. Curvature learning, an unsupervised learning approach, quantized pixels into three classes—blue, yellow, and red—representing normal, intermediate, and high cancer probability, both at baseline and after treatment. Next, a curvature learning score was calculated for each mouse using statistical measures representing variations in these three color classes across each frame from cine ultrasound images obtained during contrast administration on a given day (intra-day variability) and between pre- and post-treatment days (inter-day variability). A Wilcoxon rank-sum test compared score distributions between treated, treatment-sensitive mice and all others. There was a statistically significant difference in tumor score between the treated, treatment-sensitive group (n = 10) and all others (n = 30) (p = 0.0051). Curvature learning successfully identified treatment response, detecting changes in tumor perfusion before changes in tumor size. A similar technique could be developed for humans.

1. Introduction

The Response Evaluation Criteria in Solid Tumors (RECIST) is a widely used set of criteria to determine tumor response to therapy [1]. However, these criteria are based on tumor size, and during treatment, there is not always a measurable change in tumor size despite tumor response to therapy [2,3]. Furthermore, the first RECIST follow-up evaluation usually happens 6–8 weeks after therapy initiation at the earliest [4]. Several studies showed dynamic contrast-enhanced ultrasound (DCE-US) to be useful for monitoring treatment response, especially in patients undergoing anti-angiogenic chemotherapy [5,6]. This form of chemotherapy curtails the delivery of oxygen and nutrients to tumor cells by obliterating their vascular supply by targeting vascular endothelial growth factor (VEGF) [7]. As anti-angiogenic therapy suppresses tumor cell growth by inhibiting vascular supply, decreases in tumor size under this therapeutic regime often take longer than other forms of chemotherapy [8].

DCE-US has several advantages and a few disadvantages compared to cross-sectional modalities for tumor response evaluation, such as CT, MRI, and PET-CT [9]. Unlike CT and PET-CT, there is no ionizing radiation. DCE-US is also portable and can be performed at the patient’s bedside, is less expensive than alternative imaging modalities, and is widely available. DCE-US does not cause nephrotoxicity, an advantage for patients with renal failure [10]. DCE-US has been investigated for enhancing drug delivery into tumors and increasing tumor sensitivity to radiotherapy [11]. Hence, a physician could simultaneously diagnose and treat a tumor on the same clinical visit. Regarding disadvantages, other modalities such as CT can provide a quick survey of a large amount of patient anatomy across multiple organs, whereas ultrasound is typically more targeted. In addition, the finding under investigation needs to be accessible by ultrasound, an issue for deeper lesions, especially if they are surrounded by loops of bowel. However, DCE-US is ideally suited for early tumor response evaluation in lesions easily accessible by ultrasound, such as those in the liver and kidneys. DCE-US can detect changes in tumor perfusion as a response to treatment, allowing the early detection of cancer treatment effectiveness even before there is a change in tumor size [8].

Machine learning has recently become popular for medical applications, including in monitoring responses to cancer therapy [12,13,14]. Treatment effectiveness monitoring most commonly employs supervised machine learning. Three major categories of features that are assessed in this way are pixel-intensity features (mean, median, skewness, and kurtosis), hemodynamic parameters (peak enhancement, time to peak, mean transit time, and area under the curve), and texture/shape analysis (gray level co-occurrence matrix, wavelet features, and fractal-based features). Subsequently, feature selection techniques are used, such as least absolute shrinkage and selection operator (LASSO) regression [15] or principal component analysis (PCA) [16], and a supervised learning algorithm is subsequently run.

However, the current approach presents challenges. First, it requires manual tumor segmentation, which is time consuming even for expert readers. This is particularly burdensome for 3D dynamic contrast-enhanced ultrasound, as multiple planes must be segmented. Three-dimensional DCE-US has advantages over conventional two-dimensional DCE-US, as it is less prone to sampling error related to subtle deviations in transducer positioning [17]. This is important if the robust and reproducible quantification of tumor perfusion is desired. Since a busy ultrasound clinic may image over fifty patients per day for each radiologist, manual segmentation of tumor volumes is impractical, hindering the clinical translation of these techniques. Moreover, manual segmentation introduces inter-observer variability. Supervised machine learning requires an adequate sample size for training and validation and does not perform well with small sample sizes, leading to overfitting and lacking generalizability to external datasets [13]. The current approach also requires feature selection and is sensitive to the specific features that are chosen. Finally, the current approach is complex and requires multiple steps, including tumor segmentation, perfusion modeling, feature extraction and selection, and finally machine learning. More recently, deep learning has increased in popularity, which obviates the feature selection process. However, it is sensitive to small datasets, and its “black box” approach makes it hard to understand the reasoning behind its predictions [18]. Deep learning also typically employs supervised learning, while the algorithm presented in this paper uses an unsupervised learning approach.

We present an unsupervised approach using curvature learning that addresses several of these concerns [14,19], which is comprehensively described in Section 2.2, Section 2.3 and Section 2.4. As an unsupervised method that does not require training, it is not subject to overfitting, a major concern for small sample sizes when using supervised machine learning. This technique requires far fewer steps than traditional supervised learning, as it does not require manual segmentation, saving time and eliminating inter-observer variability, and it does not require feature selection, eliminating feature dependency. This method calculates perfusion parameters on a pixel-by-pixel basis without needing curve fitting to existing perfusion models [20,21]. The utility of this approach, which uses dimensionality reduction, was previously described for breast cancer cases undergoing breast MRI [22]. In this paper, the unsupervised curvature learning approach was applied to dynamic contrast-enhanced breast MRI scans, and the technique successfully distinguished breast tumors from normal background tissue. Therefore, the purpose of this study was to develop a new algorithm and decision support tool for the quantitative and noninvasive assessment of treatment response using contrast-enhanced ultrasound.

2. Materials and Methods

2.1. Mouse Experimental Protocol

All procedures were approved by the Administrative Panel on Laboratory Animal Care (APLAC) at Stanford University under protocol #21637, entitled “The use of contrast-enhanced ultrasound with microbubbles for applications in cancer imaging and therapeutic delivery”. Forty mice (Charles River Laboratories, Wilmington, MA, USA) implanted with colon cancer tumors were imaged via DCE-US using an EPIQ 7 ultrasound scanner with an X6-1 matrix array transducer (Philips Healthcare, Andover, MA, USA) and injected with perflutren (Definity) microbubbles (Lantheus Medical Imaging Inc., North Billerica, MA, USA), as previously described [2]. These microbubbles were diluted 1:4 in a sterile 0.9% saline solution after activation, and a 120 µL suspension was delivered in a five-second bolus at an injection rate of 24 µL/sec using an injection pump. Twenty mice were implanted with the LS174T tumor (American Type Culture Collection, Manassas, VA, USA) that responds to the anti-angiogenic drug bevacizumab (Avastin, Genentech, South San Francisco, CA, USA), a monoclonal antibody targeting VEGF. Twenty were implanted with the CT26 tumor (American Type Culture Collection, Manassas, VA, USA) that does not respond to the drug. For both the responder and non-responder groups, ten mice were treated with bevacizumab and ten served as controls that were administered saline. This resulted in four groups of ten mice: treatment-sensitive, bevacizuab-treated mice (LSBolusAV); treatment-sensitive, saline-treated mice (LSBolusCTRL); treatment-resistant, bevacizumab-treated mice (CTBolusAV); and treatment-resistant, saline-treated mice (CTBolusCTRL). In this nomenclature, the first set of capitalized letters represents the cell type (LS174T vs. CT26) and the second set of capitalized letters represents the type of treatment given (chemotherapeutic agent bevacizumab/Avastin vs. saline control). Mice administered bevacizumab can be considered “treated”, whereas mice administered saline can be considered “untreated.” Only the treatment-sensitive, bevacizumab-treated mice (LSBolusAV) were expected to respond to treatment. As shown in Figure 1, before treatment (day 0: baseline) and on post-treatment days 1, 3, 7, and 10, the tumors were imaged with DCE-US. Of note, prior research using this dataset showed that there was no change in tumor size between day 0 and post-treatment day 1 [2]. After day 3, groups with untreated tumors showed a progressive increase in size compared to the treated group.

Further description and details about the original experimental setup can be found in [2].

2.2. Dimensionality Reduction

Data analysis was performed using MATLB R2015b (The MathWorks, Inc., Natick, MA, USA). To detect the underlying data structure within DCE-US images, we applied dimensionality reduction, a method that transforms the high-dimensional DCE-US images into a lower-dimensional representation reflecting the data’s intrinsic dimensionality. This representation denotes the least amount of data that can meaningfully represent the true data structure, reducing redundant data whilst maintaining data variance [22]. The most commonly used dimensionality reduction methods are linear, assuming that the data are within a linear subspace of a higher-dimensional topological space, and these methods include principal component analysis (PCA) [23] and linear discriminant analysis (LDA) [24]. However, nonlinear dimensionality reduction techniques do not make this assumption, allowing for more complex representations of higher-dimensional data. A previous work analyzed both linear and nonlinear dimensionality reduction methods on dynamic contrast-enhanced MRI, finding that nonlinear dimensionality reduction methods were superior and that amongst the dimensionality reduction techniques, diffusion maps performed particularly well [22].

2.3. Diffusion Maps

Diffusion maps take inspiration from physics [22,25,26]. Conceptually, pixel intensities correspond to the diffusivity of different particles. Diffusion maps locate the subspace that most effectively preserves diffusion interpoint distances based on a Markov random walk on a graph of the data known as a Laplacian graph [22,26]. A Gaussian kernel function is used to estimate the weights K of the graph edges, which are defined by

$$K_{ij}=e^{{\displaystyle \frac{\Vert x_i-x_j{\Vert }^2}{2{\sigma }^2}}} i \ge 1, j\le L$$

(1)

in which L represents the number of multidimensional points and σ is a free parameter. Next, normalization is performed such that the rows of the matrix K sum to 1 by the following equation:

$$p_{ij}^{\left(t\right)}={\displaystyle \frac{K_{ij}}{{{\displaystyle \sum }}_{n=1}^L{K}_{in}}}$$

(2)

in which p represents the forward transition probability of t time steps of a random walk from one data point to another. Lastly, the diffusion distance is defined as the following:

$$D_{ij}^{\left(2\right)}={\displaystyle \sum }_{r=1}^L{\displaystyle \frac{{\left(p_{ir}^{\left(t\right)}-p_{jr}^{\left(t\right)}\right)}^2}{\psi \left(x_r\right)}}$$

(3)

$$\psi \left(x_m\right)={\displaystyle \frac{{{\displaystyle \sum }}_{j=1}^L{p}_{jm}}{{{\displaystyle \sum }}_{k=1}^L{{\displaystyle \sum }}_{j=1}^L{p}_{jk}}}$$

(4)

The high-density parts of the graph defined by the diffusion distance are weighted more heavily, whereas pairs of data points with a large forward transition probability have a smaller diffusion distance. Diffusion distance is less sensitive to noise than geodesic distance, using numerous paths within the graph to generate the embedded image. Using spectral theory, the embedded image Y can be generated by using d nontrivial eigenvectors of distance matrix D.

$$Y:x\mapsto \left\{{\lambda }_2{V}_2, {\lambda }_3{V}_3,\dots {\lambda }_n{V}_n\right\}$$

(5)

The first eigenvector V₁ of the largest eigenvalue (λ₁) is discarded, and the remaining eigenvectors are normalized by their eigenvalues.

2.4. Generation of the Embedded Image and Scattergram

The embedded vector was created by projecting pixel intensities from higher-dimensional space to a one-dimensional embedding space using diffusion maps for dimensionality reduction. The embedded image, or fused heat map, was then formed by taking this embedded vector and changing it to size m x n, the original dimensions of the image. Essentially, the higher-dimensional space has been mapped to a 2D space, creating an unfolded version of the original data manifold [27,28,29]. To create the scattergram, the higher-dimensional space is mapped to a 2D embedded space. This is then quantized into three classes for tissue characterization—blue, yellow, and red—representing normal, intermediate, and high probability for cancer, respectively. This procedure was conducted for all mice both at baseline and post-treatment. To compare groups of mice, statistical features were calculated, described in the next section. Figure 2A shows the data analysis pipeline, Figure 2B shows examples of the embedded image and the scattergram generated by the curvature learning algorithm, and Figure 2C demonstrates how statistical features from different days were used to calculate intra-day curvature learning scores and the inter-day curvature learning score.

2.5. Statistical Analysis

To compare the four groups of mice, a tumor score termed the curvature learning score (CLS) was calculated, representing the variation in the different color classes across each image frame from the ultrasound cine images obtained during contrast administration on a given day (intra-day variability) and the variation between the pre-treatment and post-treatment days (inter-day variability). For this discussion, statistical measures include, but are not limited to, “mean”, “median”, “max”, “mode”, and “min”. The mode is the most frequently occurring value in the dataset. We tested different combinations of these statistical features to calculate the CLS.

The following steps were performed. (1) Intra-day curvature learning score: on a given treatment day, first calculate the percentages for the three color-coded clusters (blue, yellow, and red) for each time point and then calculate the statistical measures over time. (2) Inter-day curvature learning score: across different days, calculate the statistical measures comparing the previously collected intra-day data. (3) Final curvature learning score: this score represents both intra-day variability and inter-day variability for a given mouse. (4) The distribution of scores between treated, treatment-sensitive mice and all other groups was compared using a Wilcoxon rank-sum test.

3. Results

A total of 40 mice were imaged in the following categories: treatment-sensitive mice in control (N = 10, receiving saline) and treated (N = 10, receiving bevacizumab) groups and treatment-resistant mice in control (10) and treated (10) groups. Each mouse was imaged using contrast-enhanced ultrasound on five different days. The curvature learning algorithm proposed in this paper, along with the pipeline described in Figure 2A, was used to characterize each pixel in the 3D image, and values were quantized into three classes: red, yellow, and blue. This procedure was repeated for all mice at baseline and on post-treatment days.

Figure 3 shows a characteristic example of a treated, treatment-sensitive mouse (LSBolusAV) on days 0 (baseline), 3, and 10. Yellow and red colors represent increasing suspicion for tumor, while dark blue represents normal tissue. In the fused heat maps on the left, there is a decrease in red and yellow signals within the tumor over time. In the bar plots on the right, there is a decrease in the yellow and red color classes over time.

Figure 4 shows an untreated, treatment-resistant mouse (CTBolusCTRL) on days 0 (baseline), 3, and 10. There is an increase in the yellow and red colors (representing tissue more suspicious for tumor) during the treatment course both in the fused heat maps and the bar plots. Once again, dark blue represents pixels with normal tissue.

To compare between the four groups of mice, a tumor score termed the curvature learning score (CLS), explained in Section 2.5, was calculated for each mouse. This score encompasses intra-day and inter-day variation in the different color classes. The distribution of scores between each group of mice was compared using a Wilcoxon rank-sum test, as shown in Figure 5. This figure compares the mice pre-treatment and three days after treatment.

Finally, we compared the first day after treatment with the pre-treatment day, as there was no significant change in tumor size on day one [2]. After examining differences in the intra-day curvature learning scores between the baseline exam and the first exam post-treatment, there was a statistically significant difference between the final curvature learning score (CLS) for the treated, treatment-sensitive group and all other groups (p = 0.0051), as shown in Figure 6. Treatment-resistant mice, as expected, did not respond to treatment, which was demonstrated quantitatively by the CLS. Treatment-sensitive mice, as expected, responded to treatment, also demonstrated quantitatively by the CLS.

4. Discussion

This study showed that there was a significant difference between treated, treatment-sensitive mice and all other groups after treatment with the chemotherapeutic agent bevacizumab. This was evident by measuring differences in tumor perfusion in a fully unsupervised fashion. By treatment day 1, there was a clear difference in tumor perfusion despite there being no significant difference in tumor size. This contrasts with traditional follow-up using the RECIST, the current basis of cancer treatment effectiveness monitoring, which typically takes months and relies on differences in tumor size.

The original study using this dataset involved the analysis of perfusion parameters [2]. For each of the forty mice, a volume of interest (VOI) was manually created, and then a time–intensity curve was generated that measured how the average signal intensity within the VOI varied with time. After curve fitting was performed to the lognormal distribution, the following perfusion parameters were obtained: peak enhancement (PE), area under the curve (AUC), and time to peak (TTP). The nonparametric paired-samples Wilcoxon rank-sum test was used to assess changes in perfusion parameters. The investigators found that for the responding group, there was a statistically significant difference in the AUC and PE between day 0 and subsequent post-treatment days (p = 0.005), while saline-treated tumors did not show such a difference. Relative changes in the AUC and PE were significantly lower for tumors treated with bevacizumab as opposed to saline on post-treatment days (p < 0.05). Interestingly, there were no differences in the TTP between mice treated with bevacizumab or saline.

However, our method has several advantages. First, it is entirely automated and does not require generating a pre-defined region of interest, which can be time consuming and technically challenging, particularly for three-dimensional data. Given the high volumes of studies that radiologists read and knowing that radiologists do not segment tumors on 3D ultrasound data, such an analysis would be impractical in a busy radiology practice. Moreover, generating a region of interest is a manual step that could introduce human error and inter-observer variability. Like the current study, most studies in the literature have focused on oncologic applications. However, they overwhelmingly used supervised learning, either with traditional machine learning or deep learning. The advantages of our unsupervised learning approach include no requirement for time-consuming manual segmentation, no dependence on a large dataset for training, no concern for overfitting, no feature dependency, and avoidance of the “black box” issue with deep learning algorithms that can make it hard to understand why models make their predictions.

In the literature, the liver has been a common application for machine learning in DCE-US. One study sought to distinguish between benign and malignant focal liver lesions using a convolutional neural network (CNN) based on the ResNet architecture [30]. The algorithm had a receiver operating characteristic area under the curve (ROC AUC) of 0.934 on the test set. Another study also sought to differentiate between benign and malignant focal liver lesions on contrast-enhanced ultrasound, combining features obtained from both spatiotemporal analysis and texture analysis performed at different time points, using several machine learning classifiers on these input features [31]. While most models (logistic regression, support vector machine, random forest, and k-nearest neighbor) performed well, a soft voting classifier combining multiple models obtained the highest balanced accuracy at 0.84. Another study used deep learning to predict microvascular invasion with hepatocellular carcinoma, a key factor related to survival [32]. The best-performing model, ResNet50 with the Bottleneck Attention Module, had an ROC AUC of 0.856. Another study used deep learning combined with radiomics for contrast-enhanced ultrasound to predict progression-free survival after radiofrequency ablation (RFA) and surgical resection (SR); this study sought to optimize the selection between those treatments for patients with early-stage hepatocellular carcinoma [33]. The c-index, which measures performance for survival models, was 0.726 for RFA and 0.741 for SR. Moreover, 17.3% of RFA patients and 27.3% of SR patients would have benefited from switching treatment, on average increasing their 2-year progression-free survival by 12% and 15%, respectively. Overall, machine learning generally performed well for several hepatic applications.

Machine learning applied to contrast-enhanced ultrasound has also been investigated for non-hepatic applications. One study used a deep learning radiomics model based on a ResNet50 backbone to distinguish between pancreatic ductal adenocarcinoma and chronic pancreatitis [34]. On two external validation cohorts, the model demonstrated an ROC AUC of 0.967 and 0.953, respectively. Another pancreatic application sought to predict the efficacy of neoadjuvant chemotherapy for pancreatic ductal adenocarcinoma [35]. Contrast-enhanced ultrasound videos were collected prior to chemotherapy initiation to predict efficacy after chemotherapy. A CNN using videos from both conventional ultrasound and contrast-enhanced ultrasound of the pancreas had an ROC AUC of 0.892, while a second CNN using videos solely of the region of interest with contrast-enhanced ultrasound had an ROC AUC of 0.908. Another clinical application used supervised machine learning with a support vector machine to classify breast tumors as benign or malignant, with an ROC AUC of 0.893 [36]. A prostate cancer application examined molecularly targeted contrast-enhanced ultrasound using a targeted anti-PSMA agent to detect prostate cancer in mouse xenografts [37]. A 3D CNN outperformed other techniques, with an accuracy over 90%. Another prostate cancer application used clinical features, time–intensity curve features, and radiomics features from conventional transrectal ultrasound and contrast-enhanced ultrasound for prostate cancer detection in the peripheral zone [38]. The model combining all these features performed best, with an ROC AUC of 0.89. Overall, the techniques in the literature generally employed traditional machine learning or deep learning using supervised learning, unlike the unsupervised learning approach presented in this paper. In addition, this study focused on early cancer diagnosis by automatically analyzing changes in tumor perfusion.

One study limitation was the sample size of 40, which could be expanded in a future study with a larger sample size. Nevertheless, there was a clear statistical separation of the responding and non-responding groups, which was evident on the first day after treatment. Moreover, obtaining well-controlled data of this variety is technically challenging, time consuming, and expensive. Another potential limitation is that while this technique worked well on this dataset, further research is needed to investigate its scalability to larger multi-institutional datasets or to other forms of cancer. However, this technique was also previously applied to dynamic contrast-enhanced breast MRI [22], which also relied on measuring changes in perfusion. Hence, we hypothesize that our technique could potentially be applied to any study in which there are measurable changes in perfusion, such as when investigating early cancer treatment effectiveness, although additional research is warranted. Another potential future direction involves expanding this study to multiple institutions, noting that external sites would need to follow a similar general protocol with respect to cell lines and treatments. This multi-institutional study may also involve analyzing other algorithms in tandem, such as traditional supervised machine learning or deep learning. An additional potential future direction would be to perform a similar such study using a molecular contrast agent targeting the VEGF receptor, namely BR55 [39]; unlike traditional DCE-US, this agent has been chemically modified to target areas of VEGF expression in vascular endothelial cells, which are upregulated in the setting of angiogenesis, and hence would demonstrate a different pattern of contrast kinetics than untargeted DCE-US. This molecular agent typically sticks to its receptor as opposed to washing out at an earlier time point. Targeted molecular ultrasound is an emerging area.

A natural future direction involves expanding this analysis to human subjects, as there may be some biological differences between human colon cancer and a human colon cancer xenograft, which may be related to differences in the tumor microenvironment. However, since tumor perfusion is not typically assessed this rapidly and this frequently after cancer treatment using contrast-enhanced ultrasound, it would require a prospective clinical study rather than a retrospective study. In fact, this could be performed as an adjunct to an existing clinical study to examine whether changes in perfusion can be detected sooner than changes in tumor size to monitor treatment effectiveness. Early determination of treatment effectiveness can spare a patient costly and ineffective chemotherapy. In agreement with the results of our study, in previous studies, unsupervised curvature learning, along with dimensionality reduction and diffusion maps, was successfully applied to cancer images [13,19,22]. Nonetheless, to the best of our knowledge, we show for the first time that curvature learning can be used in DCE-US images, and it serves as a promising, noninvasive tool to monitor early treatment changes in cancer.

5. Conclusions

Curvature learning was effective for analyzing treatment response in mouse data, and a similar technique could be used for treatment effectiveness monitoring in humans. Unlike other approaches, this could be conducted in a completely automated manner, without requiring manual delineation of a region of interest or requiring a large dataset for training a supervised machine learning algorithm. For humans, patient care would be improved if successful chemotherapy response could be assessed within days rather than waiting 2–3 months via the traditional RECIST, potentially avoiding toxic drug-related side effects or expensive, ineffective therapy.

6. Patents

Michael A. Jacobs and Alireza Akhbardeh have the patent “Multiparametric non-linear dimension reduction methods and systems related thereto”, under US Patent 9,256,966 [40].