Next Article in Journal
Mean Platelet Volume in a Series of 315 Patients with Rheumatoid Arthritis: Relationship with Disease Characteristics, including Subclinical Atherosclerosis and Cardiovascular Comorbidity
Previous Article in Journal
Feasibility of Sentinel Lymph Node Biopsy in Early-Stage Epithelial Ovarian Cancer: A Systematic Review and Meta-Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 13
Issue 20
10.3390/diagnostics13203210
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Can Whole-Body Baseline CT Radiomics Add Information to the Prediction of Best Response, Progression-Free Survival, and Overall Survival of Stage IV Melanoma Patients Receiving First-Line Targeted Therapy: A Retrospective Register Study

by
Felix Peisen
1,*,
Annika Gerken
2,
Alessa Hering
2,3,
Isabel Dahm
1,
Konstantin Nikolaou
1,4,
Sergios Gatidis
1,5,
Thomas K. Eigentler
6,7,
Teresa Amaral
6,
Jan H. Moltz
2,† and
Ahmed E. Othman
1,8,†
1
Department of Diagnostic and Interventional Radiology, Tuebingen University Hospital, Eberhard Karls University, Hoppe-Seyler-Straße 3, 72076 Tuebingen, Germany
2
Fraunhofer MEVIS, Max-von-Laue-Straße 2, 28359 Bremen, Germany
3
Diagnostic Image Analysis Group, Radboud University Medical Center (Radboudumc), Geert Grooteplein Zuid 10, 6525 GA Nijmegen, The Netherlands
4
Image-Guided and Functionally Instructed Tumor Therapies (iFIT), The Cluster of Excellence (EXC 2180), 72076 Tuebingen, Germany
5
Max Planck Institute for Intelligent Systems, Max-Planck-Ring 4, 72076 Tuebingen, Germany
6
Center of Dermato-Oncology, Department of Dermatology, Tuebingen University Hospital, Eberhard Karls University, Liebermeisterstraße 25, 72076 Tuebingen, Germany
7
Department of Dermatology, Venereology and Allergology, Charité—Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humbolt-Universität zu Berlin, Luisenstraße 2, 10117 Berlin, Germany
8
Institute of Neuroradiology, Johannes Gutenberg University Hospital Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Diagnostics 2023, 13(20), 3210; https://doi.org/10.3390/diagnostics13203210
Submission received: 27 August 2023 / Revised: 6 October 2023 / Accepted: 12 October 2023 / Published: 14 October 2023
(This article belongs to the Section Medical Imaging and Theranostics)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background: The aim of this study was to investigate whether the combination of radiomics and clinical parameters in a machine-learning model offers additive information compared with the use of only clinical parameters in predicting the best response, progression-free survival after six months, as well as overall survival after six and twelve months in patients with stage IV malignant melanoma undergoing first-line targeted therapy. Methods: A baseline machine-learning model using clinical variables (demographic parameters and tumor markers) was compared with an extended model using clinical variables and radiomic features of the whole tumor burden, utilizing repeated five-fold cross-validation. Baseline CTs of 91 stage IV malignant melanoma patients, all treated in the same university hospital, were identified in the Central Malignant Melanoma Registry and all metastases were volumetrically segmented (n = 4727). Results: Compared with the baseline model, the extended radiomics model did not add significantly more information to the best-response prediction (AUC [95% CI] 0.548 (0.188, 0.808) vs. 0.487 (0.139, 0.743)), the prediction of PFS after six months (AUC [95% CI] 0.699 (0.436, 0.958) vs. 0.604 (0.373, 0.867)), or the overall survival prediction after six and twelve months (AUC [95% CI] 0.685 (0.188, 0.967) vs. 0.766 (0.433, 1.000) and AUC [95% CI] 0.554 (0.163, 0.781) vs. 0.616 (0.271, 1.000), respectively). Conclusions: The results showed no additional value of baseline whole-body CT radiomics for best-response prediction, progression-free survival prediction for six months, or six-month and twelve-month overall survival prediction for stage IV melanoma patients receiving first-line targeted therapy. These results need to be validated in a larger cohort.

1. Background

Owing to new drugs, the therapy of melanoma patients in an advanced stage has seen dramatic improvements in recent years. In addition to checkpoint inhibition (ipilimumab (CTLA-4), nivolumab/pembrolizumab (PD-1), and their combination) [1], blocking the RAF-RAS-MEK signaling pathway with BRAF inhibitors mono or combined BRAF and MEK inhibitors (targeted therapy) has been established. However, this is possible for less than half of patients only due to the specific mutation patterns required [2,3,4,5,6]. The application of targeted therapy agents has enhanced overall survival (OS) and progression-free survival (PFS) [7,8,9]. Unfortunately, 15–20% of tumors show primary resistance to this therapy, and patients can develop resistance to treatment [10]. Clinically validated predictive markers such as lactate dehydrogenase (LDH), S100B, and tumor burden exist to identify patients who do not benefit from targeted therapy [8,11]. On the other hand, experimental biomarkers based on radiomics are the focus of recent research to improve response and survival prediction. Radiomics describes the extraction of phenotypic features from medical image data using either handcrafted features or, more recently, abstract features from deep learning and the subsequent development of imaging biomarkers using machine- or deep-learning methods [12]. While several studies have investigated the use of radiomics and machine learning to support clinical decision-making in metastatic melanoma [13,14,15,16,17,18], there is still a demand for further investigation as limited data for patients treated with targeted therapy exist and no radiomic biomarker is yet established for routine clinical use [2,17].
The aim of the study was to investigate in a machine-learning prediction model if radiomics extracted from three-dimensional segmentation of all baseline metastases, combined with clinical parameters, offers Supplementary Information compared with clinical parameters alone. Using a volumetric whole-tumor-burden segmentation approach, we aimed to extract information also from small lesions and variables such as the whole-body tumor burden. The primary endpoints were best response, progression-free survival for six months, and overall survival for six and twelve months.

2. Material and Methods

2.1. Workflow Framework

The workflow is depicted in Figure 1. In the baseline computed tomography (CT) images of the selected patient cohort, all metastatic lesion volumes were segmented by hand. A large set of radiomic features was extracted per lesion and aggregated across the lesions per patient. Two machine-learning (ML) models were trained and evaluated with a repeated five-fold cross-validation scheme. The baseline model used clinical data only; the extended model used clinical data and radiomic features. The models consisted of feature preprocessing, feature selection, training, and validation.

2.2. Patient Selection

The institutional ethics board approved the study protocol. Patients diagnosed with metastatic malignant melanoma between January 2015 and December 2018 were retrospectively identified in the melanoma registry of the department of dermatology. First-line treatment was conducted at the department of dermatology, a tertiary referral center. Inclusion criteria were metastatic malignant melanoma stage IV, first-line treatment with targeted therapy (BRAF inhibitor monotherapy or in combination with a MEK inhibitor, according to current guidelines), existing baseline CT scans, documented demographic data, follow-up data, and clinical metadata. Exclusion criteria were missing baseline imaging with CT, previous treatment with targeted therapy, or no depictable metastasis on CT imaging. For an illustration of the inclusion and exclusion process see Figure 2. For a detailed description of the final cohort, refer to Table 1.

2.3. Imaging and Metastases Segmentation

The radiomic analysis was carried out on baseline CT data, which were downloaded from the local picture archiving and communication system (PACS). Inhouse staging CTs were performed on four CT scanners and one PET-CT scanner. The inhouse whole-body staging protocol was used with a scan field from the skull base to the middle of the femur with patients laid in a supine position, arms raised above the head. Scanning was performed during the portal-venous phase after administration of body-weight-adapted contrast medium through the cubital vein. For a detailed distribution of CT vendors and inhouse scan parameters, see Table S1 of the supplement. To account for a more realistic sample and reduce sample bias, 26 baseline CTs of patients who had received their CT not inhouse at the department of diagnostic and interventional radiology, but at several external institutions, were included. Detailed information for contrast medium phase, tube current, and tube voltage are not available for these cases.
The image analysis was performed on transversely reformatted images of the portal-venous phase. Manual segmentation of all visible metastases was volumetrically performed by a radiologist (F.P., with 5 years’ experience in oncologic imaging) and verified by an experienced reader in cases of uncertainty using a four-eye principle (A.O., with 9 years’ experience in oncologic imaging). For this purpose, the anonymized data were imported into a custom-made segmentation software (SATORI, Fraunhofer MEVIS, Bremen, Germany). All clearly identifiable and measurable metastases, including 123 osseous metastases (see Table 1), were segmented to obtain the most reliable approximation of the total tumor burden. Cerebral metastases were not included in the segmentation, as the whole-body CT protocol could not replace a dedicated cranial computed tomography scan. However, the presence of brain metastases was included into the model as binary information.

2.4. Feature Extraction and Machine-Learning Model

For the radiomic feature extraction, aggregation, and development of a machine-learning model, we used a pipeline that we have described in depth in a previous publication [19]. Summing up, radiomic features were extracted for each segmented lesion using the reference standard software Pyradiomics (version 2.1.0) [12]. The lesion-wise features were aggregated per patient, to enable patient-level training and outcome prediction. In addition, the lesion count and total lesion volume were computed across all segmented lesions and per lesion type. In total, 5284 radiomic features were calculated per patient. Automatic feature selection using the fast correlation-based filter for feature selection (FCBF) method [20] was applied during training to select a small subset of features that have a high correlation with the outcome on training data and little correlation with other selected features.
A random forest (RF) was chosen as the core ML model [21] using the implementation provided in the scikit-learn Python package (version 0.24.2, ExtraTreesClassifier) [22]. Two ML models (baseline and extended) were trained for the endpoints binarized best response (CR + PR vs. SD + PD after first cycle of targeted therapy according to RECIST 1.1 [23]), progression-free survival for six months, and overall survival after six months and twelve months, respectively. The baseline model was trained on the clinical data (features listed in Figure 3). The extended model was trained on all clinical features and a subset of the aggregated radiomic features that was automatically selected with the FCBF per fold. All ML models were trained in repeated 10 × 5-fold cross-validation (CV) with random assignment of patients to the folds to obtain a reliable estimate of the overall model performance. For more in-depth technical details regarding the radiomics and ML pipeline, refer to our previous publication [19].

2.5. Model Evaluation

For the model evaluation, the area under the curve (AUC) of the receiver operating characteristic (ROC) curve was selected. Bootstrapping with 1000 samples to estimate a 95% confidence interval (CI) for the mean AUC of the 10 × 5-fold CV of each model was used [24]. The difference in performance of the two models was regarded as statistically significant if the CIs of their mean AUCs did not overlap. Additionally, we divided the cohort into low- vs. high-risk patients: A patient was classified as high-risk when the twelve-month overall survival prediction of the respective ML model was negative. The lifelines Python package (version 0.26.0) was employed to compute a Kaplan–Meier estimator for the overall survival of both groups, with a log-rank test for the statistical comparison of both groups [25].

3. Results

3.1. Demographics, Best Response, Progression-Free Survival for Six Months, and Overall Survival after Six and Twelve Months

The study cohort had a median age of 56 years and was predominantly male (59%). Superficial spreading melanoma (55%) and nodular melanoma (34%) were the most frequent histological subtypes. Baseline LDH was elevated in 42%, and baseline S100B in 51% of the patients. Table 1 provides detailed patient characteristics.

3.2. Prediction of Best Response, PFS, and Overall Survival

3.2.1. Model Evaluation

The radiomics model was numerically superior to the baseline model for the prediction of best response and PFS for six months (delta of the mean AUC for best response +0.064; delta for PFS +0.093) but not for the prediction of overall survival at six and twelve months (delta for overall survival after six months −0.094; delta for overall survival after twelve months −0.078); see Table 2. However, the difference was not significant, as there was an overlap of the confidence interval for all endpoints. Figure 4 shows the mean ROC curve for all endpoints and both models, where an overlap of the CI of the ROC curves can also be seen. Except for the baseline model for best response, all models indicated a prediction capacity (mean AUC > 0.5); however, only three models were significantly superior to random guessing (AUC 0.5 not included in CI). See Table 2.

3.2.2. Risk Stratification for Overall Survival

Depending on the twelve-month overall survival prediction in the five-fold cross-validation, applicable patients were divided into a low-risk and a high-risk group using Kaplan–Meier estimators. Subsequent log-rank tests showed a significant distinction for the baseline model (p < 0.005) but not for the extended model (p = 0.43). See Figure 5.

4. Discussion

In this study, a previously developed machine learning radiomics pipeline for pre-treatment baseline CTs of patients undergoing immunotherapy [19] was applied to a cohort undergoing targeted therapy. For the immunotherapy cohort, we described the predictive capabilities of five ML models for three clinical endpoints (response at three months and overall survival at six and twelve months), but no statistically significant difference between the baseline and extended models. We also documented a trend (p = 0.06) for risk stratification into low-risk and high-risk groups for overall survival using the extended model.
In the present study’s targeted therapy cohort, the mean AUCs were in a comparable range; however, no significant predictive capabilities of the ML models could be shown, as indicated by the large confidence intervals. For best response and PFS, the results indicated a small increase in the AUCs when adding radiomic parameters. For overall survival prediction, adding radiomic parameters led to a drop in performance. As for the immunotherapy cohort, no significant differences between the baseline and extended models could be shown for the targeted therapy cohort. The baseline model was able to differentiate high-risk patients from low-risk patients for overall survival, correlating with the results published for immunotherapy cohorts [26,27]. With the extended model, no differentiation of low-risk and high-risk patients was possible.
The results show that the Supplementary Information gained by the extended ML model for best-response prediction and PFS prediction is limited, and for the endpoints six-month and twelve-month overall survival even detrimental to the performance, as demonstrated by the low AUCs and AUC differences. Missing information from the follow-up imaging and the connected change in the lesion parameters might explain this underperformance. Gassenmaier et al. [11] investigated levels of LDH and S100B during therapy in patients with metastatic melanoma at distinct time points. Dercle et al. [28] followed a mixed delta radiomics and clinical metadata approach and investigated a patient collective with advanced melanoma treated with immunotherapy. Both authors exemplarily showed the potential of the so-called delta approach. Our aim, however, was to make a statement at the earliest point of time possible, which is the baseline CT. There are promising baseline-focused publications available for melanoma patients treated with immunotherapy [26,27] and one publication investigating the role of baseline FDG PET/CT for response prediction in a mixed immunotherapy/targeted therapy melanoma cohort [29]. To our knowledge, no publications are available for the comparison of baseline clinical data and CT radiomics for a targeted therapy melanoma cohort.
Technical challenges like model overfitting [30], varying CT acquisition parameters between vendors and institutions, as well as differences in post-processing, might also explain the low performance of the extended model. However, one must keep in mind that boards in referral centers receive patients from different institutions with CTs that have been acquired from varying sources [31,32,33]. In our eyes, a more selective approach would have therefore not been beneficial for clinical transferability of the results. Nevertheless, the AUCs of all our extended ML models were not very high (<0.7), which indicates that there was no stable biomarker. This matches with previous results from a large cohort undergoing first-line immunotherapy [19].
The present study included a cohort of patients with prospective documentation of clinical metadata and long-term follow-up in a registry. The sample size was comparable to other radiomic studies investigating melanoma patients treated with targeted therapy [34] and immunotherapy [13,14,35,36]; however, the number of segmented lesions was larger in our cohort due to the whole-body segmentation approach [16]. The cohort consisted of a first-line treatment collective and exhibited the prescribed trend that patients with BRAF(+) are often younger and present with superficial spreading tumors or tumors with nodular histology, developing in anatomical regions without chronic sun damage [10]. The cohort showed a typical response distribution to targeted therapy. Median overall survival was in the reported range of a stage IV melanoma cohort treated with targeted therapy [7]. The random forest model was therefore developed and tested in a real-life cohort. By including different CT vendors and image data sources, selection bias was reduced. The whole-body segmentation approach was chosen to account for a potential loss of information from non-target lesions, which are usually not included in RECIST-based studies. All visible metastases in the baseline CTs were volumetrically segmented to gain the maximum information of the whole tumor load and to create as much information output as possible for the radiomics analysis as a benchmark for predictive performance.
Even though the study used a large number of segmented metastases and a similar sample size compared to other radiomic studies, an extended collective with more independent samples would have been of value for the development of a radiomics ML model. The applied study protocol might have underestimated the true total tumor burden in some patients. Other modalities like FDG-PET/CT are more sensitive to the detection of certain types of metastases, such as osseous lesions. However, FDG-PET/CT is not the standard modality for staging in all melanoma patients yet, leading to smaller numbers of available subjects. We therefore decided to focus on widespread available CT and tried to compensate for the reduced sensitivity by consulting follow-up CTs and other staging modalities, if applicable, to identify as many lesions on baseline imaging as possible. Because there was a very large number of lesions and the outlining was carried out by hand, the volumetric segmentation was executed only once by just one reader (F.P.). Supervision by an experienced radiologist (A.O.) was given; however, a second reading and a second reader are missing. An assessment of inter- and intra-reader variability was therefore not possible. There is partially missing information about CT acquisition parameters because the study included external imaging sources to reduce selection bias, guarantee a realistic sample as requested by current radiological guidelines, and not further reduce the cohort size [37]. An external validation cohort is lacking. We aimed to compensate for that limitation through multiple cross-validations, an established method used in prior studies [38], and believe that an external validation cohort would not have delivered additional information, as the results are pointing toward a failure of the tested method.
The results of the study mainly point toward a failure of the investigated method. However, through our study design and despite the negative results, we hope to address the recently discussed bias toward the publication of positive results in radiomic studies, lacking a comparison with a non-radiomic approach, as described by Kocak et al. [39].

5. Conclusions

The study applied a previously developed radiomics machine-learning pipeline to a stage IV melanoma patient cohort undergoing first-line targeted therapy. In the selected sample, it indicated no added value of radiomics in baseline CT imaging for best-response prediction, progression-free survival prediction for six months, or six-month and twelve-month overall survival prediction. The resulting AUCs remained low, even though all visible metastases in the whole body were volumetrically segmented. To improve the capacity of the current and other models, it is necessary to include larger cohorts with more independent samples and apply techniques combining different biomarkers and radiomic features.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics13203210/s1, Table S1: CT scanner vendors and scan parameters.

Author Contributions

Conceptualization, F.P., A.G., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Methodology, F.P., A.G., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Software, F.P., A.G., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Validation, F.P., A.G., A.H., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Formal analysis, F.P., A.G., J.H.M. and A.E.O.; Investigation, F.P., A.G., A.H., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Resources, F.P., A.G., A.H., I.D., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Data curation, F.P., A.G., A.H., I.D., T.K.E., T.A., J.H.M. and A.E.O.; Writing—original draft preparation, F.P., A.G., T.K.E., J.H.M. and A.E.O.; Writing—review and editing, F.P., A.G., A.H., I.D., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Visualization, F.P., A.G. and J.H.M.; Supervision, S.G., T.K.E., J.H.M. and A.E.O.; Project administration, F.P., A.G., J.H.M. and A.E.O.; Funding acquisition, F.P., S.G., T.K.E., T.A., J.H.M. and A.E.O. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the SPP2177 program of the German Research Foundation (Deutsche Forschungsgemeinschaft, ‘DFG’), project number #428216905. The DFG was not involved in the design of the study; the collection, analysis, and interpretation of data; or in writing the manuscript.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Ethics Committee of the Medical Faculty Eberhard-Karls-University Tübingen (protocol code 092/2019BO2, 21 February 2019).

Informed Consent Statement

Patient consent was waived by the Institutional Ethics Committee due to the retrospective study design.

Data Availability Statement

Data may be made available after a reasonable and well justified request to Felix Peisen. Data cannot, however, be made freely available to the public, due to privacy regulations. Codes and materials used in this study may be made available for purposes of reproducing or extending the analysis, pending materials transfer agreements.

Acknowledgments

The authors would like to thank Max Westphal for assistance with the statistical analysis and Andreas Daul for assistance with data curation. We acknowledge support from the Open Access Publication Fund of the University of Tübingen.

Conflicts of Interest

The authors declare that they have no competing interests.

Abbreviations

ALMacral lentiginous melanoma
AUCarea under the curve
BRAFv-Raf murine sarcoma viral oncogene homolog B1
CIconfidence interval
CMMRCentral Malignant Melanoma Registry
CRcomplete response
CTcomputed tomography
CTLA-4 cytotoxic T-lymphocyte-associated protein 4
CVcross-validation
FCBFfast correlation-based filter for feature selection
IQRinterquartile range
LDHlactate dehydrogenase
LMMlentigo maligna melanoma
LoGLaplacian of Gaussian
MEKmitogen-activated protein kinase
MLmachine learning
mRMRminimum redundancy maximum relevance feature selection
nnumber
NMnodular melanoma
OSoverall survival
PACSpicture archiving and communication system
PDprogressive disease
PD-1programmed death 1
PETpositron emission tomography
PFSprogression-free survival
PRpartial response
RAFrapidly accelerated fibrosarcoma
RASrat sarcoma
RECISTresponse evaluation criteria in solid tumors
ROCreceiver operating characteristic
SDstable disease
SSMsuperficial spreading melanoma

References

  1. Robert, C.; Long, G.V.; Brady, B.; Dutriaux, C.; Maio, M.; Mortier, L.; Hassel, J.C.; Rutkowski, P.; McNeil, C.; Kalinka-Warzocha, E.; et al. Nivolumab in previously untreated melanoma without BRAF mutation. N. Engl. J. Med. 2015, 372, 320–330. [Google Scholar] [CrossRef] [PubMed]
  2. Schadendorf, D.; van Akkooi, A.C.J.; Berking, C.; Griewank, K.G.; Gutzmer, R.; Hauschild, A.; Stang, A.; Roesch, A.; Ugurel, S. Melanoma. Lancet 2018, 392, 971–984. [Google Scholar] [CrossRef] [PubMed]
  3. Larkin, J.; Ascierto, P.A.; Dreno, B.; Atkinson, V.; Liszkay, G.; Maio, M.; Mandala, M.; Demidov, L.; Stroyakovskiy, D.; Thomas, L.; et al. Combined vemurafenib and cobimetinib in BRAF-mutated melanoma. N. Engl. J. Med. 2014, 371, 1867–1876. [Google Scholar] [CrossRef]
  4. Long, G.V.; Stroyakovskiy, D.; Gogas, H.; Levchenko, E.; de Braud, F.; Larkin, J.; Garbe, C.; Jouary, T.; Hauschild, A.; Grob, J.J.; et al. Dabrafenib and trametinib versus dabrafenib and placebo for Val600 BRAF-mutant melanoma: A multicentre, double-blind, phase 3 randomised controlled trial. Lancet 2015, 386, 444–451. [Google Scholar] [CrossRef] [PubMed]
  5. Robert, C.; Karaszewska, B.; Schachter, J.; Rutkowski, P.; Mackiewicz, A.; Stroiakovski, D.; Lichinitser, M.; Dummer, R.; Grange, F.; Mortier, L.; et al. Improved overall survival in melanoma with combined dabrafenib and trametinib. N. Engl. J. Med. 2015, 372, 30–39. [Google Scholar] [CrossRef] [PubMed]
  6. Dummer, R.; Ascierto, P.A.; Gogas, H.J.; Arance, A.; Mandala, M.; Liszkay, G.; Garbe, C.; Schadendorf, D.; Krajsova, I.; Gutzmer, R.; et al. Encorafenib plus binimetinib versus vemurafenib or encorafenib in patients with BRAF-mutant melanoma (COLUMBUS): A multicentre, open-label, randomised phase 3 trial. Lancet Oncol. 2018, 19, 603–615. [Google Scholar] [CrossRef]
  7. Amann, V.C.; Ramelyte, E.; Thurneysen, S.; Pitocco, R.; Bentele-Jaberg, N.; Goldinger, S.M.; Dummer, R.; Mangana, J. Developments in targeted therapy in melanoma. Eur. J. Surg. Oncol. 2017, 43, 581–593. [Google Scholar] [CrossRef]
  8. Timar, J.; Ladanyi, A. Molecular Pathology of Skin Melanoma: Epidemiology, Differential Diagnostics, Prognosis and Therapy Prediction. Int. J. Mol. Sci. 2022, 23, 5384. [Google Scholar] [CrossRef]
  9. Guo, W.; Wang, H.; Li, C. Signal pathways of melanoma and targeted therapy. Signal Transduct. Target. Ther. 2021, 6, 424. [Google Scholar] [CrossRef]
  10. Czarnecka, A.M.; Bartnik, E.; Fiedorowicz, M.; Rutkowski, P. Targeted Therapy in Melanoma and Mechanisms of Resistance. Int. J. Mol. Sci. 2020, 21, 4576. [Google Scholar] [CrossRef]
  11. Gassenmaier, M.; Lenders, M.M.; Forschner, A.; Leiter, U.; Weide, B.; Garbe, C.; Eigentler, T.K.; Wagner, N.B. Serum S100B and LDH at Baseline and During Therapy Predict the Outcome of Metastatic Melanoma Patients Treated with BRAF Inhibitors. Target. Oncol. 2021, 16, 197–205. [Google Scholar] [CrossRef] [PubMed]
  12. Van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.C.; Pieper, S.; Aerts, H. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef]
  13. Durot, C.; Mule, S.; Soyer, P.; Marchal, A.; Grange, F.; Hoeffel, C. Metastatic melanoma: Pretreatment contrast-enhanced CT texture parameters as predictive biomarkers of survival in patients treated with pembrolizumab. Eur. Radiol. 2019, 29, 3183–3191. [Google Scholar] [CrossRef] [PubMed]
  14. Trebeschi, S.; Drago, S.G.; Birkbak, N.J.; Kurilova, I.; Calin, A.M.; Delli Pizzi, A.; Lalezari, F.; Lambregts, D.M.J.; Rohaan, M.W.; Parmar, C.; et al. Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers. Ann. Oncol. 2019, 30, 998–1004. [Google Scholar] [CrossRef] [PubMed]
  15. Basler, L.; Gabrys, H.S.; Hogan, S.A.; Pavic, M.; Bogowicz, M.; Vuong, D.; Tanadini-Lang, S.; Forster, R.; Kudura, K.; Huellner, M.W.; et al. Radiomics, Tumor Volume, and Blood Biomarkers for Early Prediction of Pseudoprogression in Patients with Metastatic Melanoma Treated with Immune Checkpoint Inhibition. Clin. Cancer Res. 2020, 26, 4414–4425. [Google Scholar] [CrossRef]
  16. Guerrisi, A.; Loi, E.; Ungania, S.; Russillo, M.; Bruzzaniti, V.; Elia, F.; Desiderio, F.; Marconi, R.; Solivetti, F.M.; Strigari, L. Novel cancer therapies for advanced cutaneous melanoma: The added value of radiomics in the decision making process—A systematic review. Cancer Med. 2020, 9, 1603–1612. [Google Scholar] [CrossRef]
  17. Guerrisi, A.; Russillo, M.; Loi, E.; Ganeshan, B.; Ungania, S.; Desiderio, F.; Bruzzaniti, V.; Falcone, I.; Renna, D.; Ferraresi, V.; et al. Exploring CT Texture Parameters as Predictive and Response Imaging Biomarkers of Survival in Patients With Metastatic Melanoma Treated With PD-1 Inhibitor Nivolumab: A Pilot Study Using a Delta-Radiomics Approach. Front. Oncol. 2021, 11, 704607. [Google Scholar] [CrossRef]
  18. Wang, Z.L.; Mao, L.L.; Zhou, Z.G.; Si, L.; Zhu, H.T.; Chen, X.; Zhou, M.J.; Sun, Y.S.; Guo, J. Pilot Study of CT-Based Radiomics Model for Early Evaluation of Response to Immunotherapy in Patients With Metastatic Melanoma. Front. Oncol. 2020, 10, 1524. [Google Scholar] [CrossRef]
  19. Peisen, F.; Hansch, A.; Hering, A.; Brendlin, A.S.; Afat, S.; Nikolaou, K.; Gatidis, S.; Eigentler, T.; Amaral, T.; Moltz, J.H.; et al. Combination of Whole-Body Baseline CT Radiomics and Clinical Parameters to Predict Response and Survival in a Stage-IV Melanoma Cohort Undergoing Immunotherapy. Cancers 2022, 14, 2992. [Google Scholar] [CrossRef]
  20. Yu, L.; Liu, H. Feature selection for high-dimensional data: A fast correlation-based filter solution. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA, 21–24 August 2003; Volume 2, pp. 856–863. [Google Scholar]
  21. Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
  22. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
  23. Eisenhauer, E.A.; Therasse, P.; Bogaerts, J.; Schwartz, L.H.; Sargent, D.; Ford, R.; Dancey, J.; Arbuck, S.; Gwyther, S.; Mooney, M.; et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur. J. Cancer 2009, 45, 228–247. [Google Scholar] [CrossRef] [PubMed]
  24. Tsamardinos, I.; Greasidou, E.; Borboudakis, G. Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation. Mach. Learn. 2018, 107, 1895–1922. [Google Scholar] [CrossRef]
  25. Davidson-Pilon, C. lifelines: Survival analysis in Python. J. Open Source Softw. 2019, 4, 1317. [Google Scholar] [CrossRef]
  26. Awada, G.; Jansen, Y.; Schwarze, J.K.; Tijtgat, J.; Hellinckx, L.; Gondry, O.; Vermeulen, S.; Warren, S.; Schats, K.; van Dam, P.J.; et al. A Comprehensive Analysis of Baseline Clinical Characteristics and Biomarkers Associated with Outcome in Advanced Melanoma Patients Treated with Pembrolizumab. Cancers 2021, 13, 168. [Google Scholar] [CrossRef]
  27. Weide, B.; Martens, A.; Hassel, J.C.; Berking, C.; Postow, M.A.; Bisschop, K.; Simeone, E.; Mangana, J.; Schilling, B.; Di Giacomo, A.M.; et al. Baseline Biomarkers for Outcome of Melanoma Patients Treated with Pembrolizumab. Clin. Cancer Res. 2016, 22, 5487–5496. [Google Scholar] [CrossRef]
  28. Dercle, L.; Zhao, B.; Gonen, M.; Moskowitz, C.S.; Firas, A.; Beylergil, V.; Connors, D.E.; Yang, H.; Lu, L.; Fojo, T.; et al. Early Readout on Overall Survival of Patients With Melanoma Treated With Immunotherapy Using a Novel Imaging Analysis. JAMA Oncol. 2022, 8, 385–392. [Google Scholar] [CrossRef]
  29. Liberini, V.; Rubatto, M.; Mimmo, R.; Passera, R.; Ceci, F.; Fava, P.; Tonella, L.; Polverari, G.; Lesca, A.; Bello, M.; et al. Predictive Value of Baseline [18F]FDG PET/CT for Response to Systemic Therapy in Patients with Advanced Melanoma. J. Clin. Med. 2021, 10, 4994. [Google Scholar] [CrossRef]
  30. Fusco, R.; Granata, V.; Grazzini, G.; Pradella, S.; Borgheresi, A.; Bruno, A.; Palumbo, P.; Bruno, F.; Grassi, R.; Giovagnoni, A.; et al. Radiomics in medical imaging: Pitfalls and challenges in clinical management. Jpn. J. Radiol. 2022, 40, 919–929. [Google Scholar] [CrossRef]
  31. Lubner, M.G.; Smith, A.D.; Sandrasegaran, K.; Sahani, D.V.; Pickhardt, P.J. CT Texture Analysis: Definitions, Applications, Biologic Correlates, and Challenges. Radiographics 2017, 37, 1483–1503. [Google Scholar] [CrossRef]
  32. Mackin, D.; Fave, X.; Zhang, L.; Fried, D.; Yang, J.; Taylor, B.; Rodriguez-Rivera, E.; Dodge, C.; Jones, A.K.; Court, L. Measuring Computed Tomography Scanner Variability of Radiomics Features. Invest. Radiol. 2015, 50, 757–765. [Google Scholar] [CrossRef] [PubMed]
  33. Berenguer, R.; Pastor-Juan, M.D.R.; Canales-Vazquez, J.; Castro-Garcia, M.; Villas, M.V.; Mansilla Legorburo, F.; Sabater, S. Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition Parameters. Radiology 2018, 288, 407–415. [Google Scholar] [CrossRef]
  34. Colen, R.R.; Ologun, G.O.; Zinn, P.; AK, M.; Arora, R.; Burton, E.M.; Glitza, I.C.; Tawbi, H.A.-H.; Patel, S.P.; Diab, A.; et al. Radiomic signatures to predict response to targeted therapy and immune checkpoint blockade in melanoma patients (pts) on neoadjuvant therapy. J. Clin. Oncol. 2020, 38, 10067. [Google Scholar] [CrossRef]
  35. Sun, R.; Limkin, E.J.; Vakalopoulou, M.; Dercle, L.; Champiat, S.; Han, S.R.; Verlingue, L.; Brandao, D.; Lancia, A.; Ammari, S.; et al. A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: An imaging biomarker, retrospective multicohort study. Lancet Oncol. 2018, 19, 1180–1191. [Google Scholar] [CrossRef]
  36. Smith, A.D.; Gray, M.R.; del Campo, S.M.; Shlapak, D.; Ganeshan, B.; Zhang, X.; Carson, W.E., 3rd. Predicting Overall Survival in Patients With Metastatic Melanoma on Antiangiogenic Therapy and RECIST Stable Disease on Initial Posttherapy Images Using CT Texture Analysis. AJR Am. J. Roentgenol. 2015, 205, W283–W293. [Google Scholar] [CrossRef]
  37. Bluemke, D.A.; Moy, L.; Bredella, M.A.; Ertl-Wagner, B.B.; Fowler, K.J.; Goh, V.J.; Halpern, E.F.; Hess, C.P.; Schiebler, M.L.; Weiss, C.R. Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers-From the Radiology Editorial Board. Radiology 2020, 294, 487–489. [Google Scholar] [CrossRef]
  38. Kourou, K.; Exarchos, T.P.; Exarchos, K.P.; Karamouzis, M.V.; Fotiadis, D.I. Machine learning applications in cancer prognosis and prediction. Comput. Struct. Biotechnol. J. 2015, 13, 8–17. [Google Scholar] [CrossRef]
  39. Kocak, B.; Bulut, E.; Bayrak, O.N.; Okumus, A.A.; Altun, O.; Borekci Arvas, Z.; Kavukoglu, I. NEgatiVE results in Radiomics research (NEVER): A meta-research study of publication bias in leading radiology journals. Eur. J. Radiol. 2023, 163, 110830. [Google Scholar] [CrossRef]
Diagnostics 13 03210 g001
Figure 1. Framework of the machine-learning workflow. Abbreviations (ML: machine learning; CV: cross-validation).
Figure 1. Framework of the machine-learning workflow. Abbreviations (ML: machine learning; CV: cross-validation).
Diagnostics 13 03210 g001
Diagnostics 13 03210 g002
Figure 2. Patient selection.
Figure 2. Patient selection.
Diagnostics 13 03210 g002
Diagnostics 13 03210 g003
Figure 3. Schematic workflow of the radiomic analysis. Top yellow box: Examples of manual segmentations in axial reformatted CT slices in portal venous contrast medium phase. Second yellow box from top: radiomic features extracted per lesion. Third yellow box from top: Radiomic features aggregated per patient. Green box: machine-learning process. Bottom yellow box: Clinical feature set. Abbreviations: AUC (area under the curve), BRAF (v-Raf murine sarcoma viral oncogene homolog B1), FCBF (fast correlation-based filter), LDH (lactate dehydrogenase), LoG (Laplacian of Gaussian).
Figure 3. Schematic workflow of the radiomic analysis. Top yellow box: Examples of manual segmentations in axial reformatted CT slices in portal venous contrast medium phase. Second yellow box from top: radiomic features extracted per lesion. Third yellow box from top: Radiomic features aggregated per patient. Green box: machine-learning process. Bottom yellow box: Clinical feature set. Abbreviations: AUC (area under the curve), BRAF (v-Raf murine sarcoma viral oncogene homolog B1), FCBF (fast correlation-based filter), LDH (lactate dehydrogenase), LoG (Laplacian of Gaussian).
Diagnostics 13 03210 g003
Diagnostics 13 03210 g004
Figure 4. Mean ROC curves with 95% confidence interval for the true positive rate computed by bootstrapping the 10 × 5-fold CV.
Figure 4. Mean ROC curves with 95% confidence interval for the true positive rate computed by bootstrapping the 10 × 5-fold CV.
Diagnostics 13 03210 g004
Diagnostics 13 03210 g005
Figure 5. Kaplan–Meier estimators for low-risk and high-risk groups based on the predicted 12-month overall survival. p-values from log-rank tests are given for the separation of both risk groups.
Figure 5. Kaplan–Meier estimators for low-risk and high-risk groups based on the predicted 12-month overall survival. p-values from log-rank tests are given for the separation of both risk groups.
Diagnostics 13 03210 g005
Table 1. Patients’ characteristics.
Table 1. Patients’ characteristics.
Clinical Data
Age (years) [median, (IQR)] 56 (21.5)
Gender (female) [n, %] 37 (41%)
Localization of primary tumor [n, %]Head/neck11 (12%)
Torso39 (43%)
Upper extremity5 (5%)
Lower extremity19 (21%)
n/a17 (19%)
Histological subtype [n, %]SSM50 (55%)
NM31 (34%)
LMM1 (1%)
ALM1 (1%)
n/a8 (9%)
BRAF V600E mutation status [n, %]BRAF wildtype0 (0%)
BRAF mutation91 (100%)
Baseline LDH [n, %]Normal (<250 U/L)33 (36%)
Elevated (≥250 U/L)42 (46%)
n/a16 (18%)
Baseline S100B [n, %]Normal (<0.1 µg/L)23 (25%)
Elevated (≥0.1 µg/L)51 (56%)
n/a17 (19%)
Number of metastatic organs [n, %]1–370 (77%)
>321 (23%)
Presence of cerebral metastases [n, %] 27 (30%)
Presence of hepatic metastases [n, %] 29 (32%)
Targeted therapy [n, %]Dabrafenib4 (4%)
Dabrafenib + trametinib43 (47%)
Vemurafenib1 (1%)
Vemurafenib + cobimetinib36 (40%)
Encorafenib1 (1%)
Encorafenib + binimetinib6 (7%)
Lesion counts [n]All4727
Lung2006
Liver753
Soft tissue/skin1155
Lymph nodes379
Skeletal123
Spleen99
Other212
Patient outcome
Best response (RECIST 1.1) [n, %]CR11 (12%)
PR43 (47%)
SD15 (16%)
PD19 (21%)
n/a3 (3%)
Progression-free survival for 6 months [n, %]Yes36 (40%)
No44 (48%)
n/a11 (12%)
Overall survival after 6 months [n, %]Yes69 (76%)
No17 (19%)
n/a5 (5%)
Overall survival after 12 months [n, %]Yes37 (41%)
No34 (37%)
n/a20 (22%)
Overall survival (months) [median, (95%CI)] 26.2 (20.8–31.6)
Abbreviations: ALM, acral lentiginous melanoma; CR, complete response; IQR, interquartile range; LDH, lactate dehydrogenase; LMM, lentigo maligna melanoma; n/a, not available; NM, nodular melanoma;; PD, progressive disease; PR, partial response; RECIST, response evaluation criteria in solid tumors; SD, stable disease; SSM, superficial spreading melanoma.
Table 2. Number of cases with class distributions and mean AUC from a 10 × 5-fold CV and 95% confidence interval computed by bootstrapping the 10 × 5-fold CV. Class 0 best response = PD/SD; Class 1 best response = PR/CR. Class 0 PFS/OS = no; Class 1 PFS/OS = yes).
Table 2. Number of cases with class distributions and mean AUC from a 10 × 5-fold CV and 95% confidence interval computed by bootstrapping the 10 × 5-fold CV. Class 0 best response = PD/SD; Class 1 best response = PR/CR. Class 0 PFS/OS = no; Class 1 PFS/OS = yes).
Binary Endpoint
Best ResponsePFS for 6 MonthsOS at 6 MonthsOS at 12 Months
n cases, (class 0, class 1)88 (34, 54)80 (44, 36)86 (17, 69)71 (34, 37)
Baseline model (clinical features), [AUC (95% CI)]0.460 (0.362, 0.559)0.586 (0.479, 0.693)0.742 (0.631, 0.853)0.600 (0.489, 0.709)
Extended model (clinical and radiomic features), [AUC (95% CI)]0.524 (0.431, 0.615)0.679 (0.579, 0.774)0.648 (0.524, 0.769)0.522 (0.424, 0.625)
Abbreviations: AUC, area under the curve; CI, confidence interval; CV, cross-validation; n, number; CR, complete response; OS, overall survival; PD, progressive disease; PR, partial response; SD, stable disease.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Peisen, F.; Gerken, A.; Hering, A.; Dahm, I.; Nikolaou, K.; Gatidis, S.; Eigentler, T.K.; Amaral, T.; Moltz, J.H.; Othman, A.E. Can Whole-Body Baseline CT Radiomics Add Information to the Prediction of Best Response, Progression-Free Survival, and Overall Survival of Stage IV Melanoma Patients Receiving First-Line Targeted Therapy: A Retrospective Register Study. Diagnostics 2023, 13, 3210. https://doi.org/10.3390/diagnostics13203210

AMA Style

Peisen F, Gerken A, Hering A, Dahm I, Nikolaou K, Gatidis S, Eigentler TK, Amaral T, Moltz JH, Othman AE. Can Whole-Body Baseline CT Radiomics Add Information to the Prediction of Best Response, Progression-Free Survival, and Overall Survival of Stage IV Melanoma Patients Receiving First-Line Targeted Therapy: A Retrospective Register Study. Diagnostics. 2023; 13(20):3210. https://doi.org/10.3390/diagnostics13203210

Chicago/Turabian Style

Peisen, Felix, Annika Gerken, Alessa Hering, Isabel Dahm, Konstantin Nikolaou, Sergios Gatidis, Thomas K. Eigentler, Teresa Amaral, Jan H. Moltz, and Ahmed E. Othman. 2023. "Can Whole-Body Baseline CT Radiomics Add Information to the Prediction of Best Response, Progression-Free Survival, and Overall Survival of Stage IV Melanoma Patients Receiving First-Line Targeted Therapy: A Retrospective Register Study" Diagnostics 13, no. 20: 3210. https://doi.org/10.3390/diagnostics13203210

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop