Next Article in Journal
Nivolumab as Second-Line Therapy Improves Survival in Patients with Unresectable Hepatocellular Carcinoma
Next Article in Special Issue
Ultrasound Combination to Improve the Efficacy of Current Boron Neutron Capture Therapy for Head and Neck Cancer
Previous Article in Journal
Outcome of Patients with Locally Advanced Rectal Cancer Pursuing Non-Surgical Strategy in National Cancer Database
Previous Article in Special Issue
MRI for Differentiation between HPV-Positive and HPV-Negative Oropharyngeal Squamous Cell Carcinoma: A Systematic Review
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 16
Issue 12
10.3390/cancers16122195
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Development and Validation of Prognostic Models Using Radiomic Features from Pre-Treatment Positron Emission Tomography (PET) Images in Head and Neck Squamous Cell Carcinoma (HNSCC) Patients

by
Mahima Merin Philip
1,
Jessica Watts
2,
Fergus McKiddie
2,
Andy Welch
3 and
Mintu Nath
1,*
1
Institute of Applied Health Sciences, University of Aberdeen, Aberdeen AB25 2ZD, UK
2
National Health Service Grampian, Aberdeen AB15 6RE, UK
3
Institute of Education in Healthcare and Medical Sciences, University of Aberdeen, Aberdeen AB25 2ZD, UK
*
Author to whom correspondence should be addressed.
Cancers 2024, 16(12), 2195; https://doi.org/10.3390/cancers16122195
Submission received: 17 May 2024 / Accepted: 4 June 2024 / Published: 11 June 2024
(This article belongs to the Special Issue Radiotherapy for Head and Neck Squamous Cell Carcinoma)
Browse Figures
Versions Notes

Abstract

:

Simple Summary

Time-to-event analysis holds significant relevance for diseases like cancer since accurate disease prognosis is crucial for better patient management and for personalizing treatment. In recent years, survival models using machine learning (ML)-based tools have shown promise in cancer prognosis. We compared four survival models in the ML framework to predict adverse outcomes—all-cause mortality (ACM), locoregional recurrence/residual disease (LR), and distant metastasis (DM)—in head and neck cancer patients. Using radiomic features from pre-treatment positron emission tomography (PET) images, we assessed the performance of these models in an external independent validation cohort. The best-performing model for each outcome was identified based on the highest concordance index and the lowest error in training data. The penalized Cox model for ACM and DM and the random forest model for LR showed promising results. Further training and validation of these models in a larger cohort is required for clinical implementation.

Abstract

High-dimensional radiomics features derived from pre-treatment positron emission tomography (PET) images offer prognostic insights for patients with head and neck squamous cell carcinoma (HNSCC). Using 124 PET radiomics features and clinical variables (age, sex, stage of cancer, site of cancer) from a cohort of 232 patients, we evaluated four survival models—penalized Cox model, random forest, gradient boosted model and support vector machine—to predict all-cause mortality (ACM), locoregional recurrence/residual disease (LR) and distant metastasis (DM) probability during 36, 24 and 24 months of follow-up, respectively. We developed models with five-fold cross-validation, selected the best-performing model for each outcome based on the concordance index (C-statistic) and the integrated Brier score (IBS) and validated them in an independent cohort of 102 patients. The penalized Cox model demonstrated better performance for ACM (C-statistic = 0.70, IBS = 0.12) and DM (C-statistic = 0.70, IBS = 0.08) while the random forest model displayed better performance for LR (C-statistic = 0.76, IBS = 0.07). We conclude that the ML-based prognostic model can aid clinicians in quantifying prognosis and determining effective treatment strategies, thereby improving favorable outcomes in HNSCC patients.

1. Introduction

Head and neck squamous cell carcinoma (HNSCC) is a common cancer worldwide, with nearly 900,000 new cases and 500,000 deaths in 2020 alone [1]. Despite advances in medical sciences, the five-year survival rate for HNSCC has not improved over the past three decades. More than 50% of HNSCC patients experience recurrence and distant metastasis within three years of diagnosis. In cases of locally advanced disease, HNSCC has poor prognosis, with a 5-year overall survival rate of less than 50%. To aggravate the issue, treatment options to treat recurrent and metastatic HNSCC are limited compared to other malignancies owing to the disease’s complexity and heterogeneity, contributing to its poor prognosis [2,3,4,5]. Chemotherapy, chemoradiotherapy, targeted therapy and immunotherapy for HNSCC treatments show varying results depending on the age, stage of disease, comorbidities and previous treatment history. The current clinical trials are exploring the promises of molecular targeted therapy and immunotherapy as treatment strategies in HNSCC patients [5]. Given these challenges, early identification of patients with low survival rates is crucial in providing suitable treatment regimens for these patients [6]. However, the accurate prediction of disease prognosis is a challenging task for clinicians due to the complexity of each patient case and increased variability in individual prognostic factors [7,8].
Positron emission tomography (PET) imaging is superior to other imaging modalities in identifying locoregional nodal involvement or distant metastasis [9]. PET radiomic features from primary tumors along with clinical variables may be useful in developing robust prognostic models, predicting and stratifying disease risks, and applying patient-specific treatment strategies [10]. Radiomic features allow the quantification of tumor phenotype and provide insight into various aspects of the disease such as the grade of a tumor, histologic and genetic subtype and the predicted outcome [11]. Combining radiomic features with clinical parameters demonstrated complementary predictive value for adverse outcomes in patients with HNSCC [12]. Additionally, identifying predictors associated with outcomes is critical to provide insights into disease development and progression mechanisms [13].
To develop a robust prognostic model in the presence of censored data with the capability to predict time-to-event outcomes, such as all-cause mortality (ACM), locoregional residual/recurrent disease (LR) or distant metastasis (DM), survival models are considered as the most suitable approach [14]. In survival analysis, a key concern is to deal with censored events where the occurrence of the event of interest is not fully observed for all the subjects by the end of the study. When some subjects do not experience the event of interest or are lost to follow-up during the study period, they are labeled as right-censored events [15]. While the standard Cox proportional hazard (CPH) model represents the conventional choice for survival data with censored events [13], it cannot handle large-scale data efficiently [14]. However, numerous advances have been made to improve survival prediction models, leveraging higher volume and dimension of data with enhanced computational feasibilities. Among these models, machine learning (ML) algorithms have shown promising results [7]. ML models can effectively handle high-dimensional, right-censored and heterogeneous data [14]. The application of ML techniques has led to a notable enhancement in cancer outcome prediction, with improvements ranging from 15% to 20% [8]. ML models also offer significant benefits in identifying patterns and discovering relationships between the predictive features and the outcome probabilities [8]. An array of machine learning methods, including kernel methods, gradient boosting and neural networks, are currently available to model survival outcomes [13].
To our knowledge, limited literature has addressed the suitability of ML models with external validations for time-to-event modelling of adverse outcomes in HNSCC patients using pre-treatment FDG (fluorodeoxyglucose)-PET images [16]. Therefore, the objectives of this study were: (a) to develop and compare the performance of different ML algorithms and validate using an external independent cohort, (b) to identify the best-performing model to estimate the survival probability and (c) to explore features influencing the performance of best models for different outcomes in HNSCC patients.

2. Materials and Methods

2.1. Patient Data

CheckList for EvaluAtion of Radiomics research (CLEAR) was used for reporting the study [17]. The details of the checklist are available in the supplementary material Table S1. Data for this retrospective study were available from the public database, The Cancer Imaging Archive (TCIA). Ethics approval was not required for this study. Two different datasets available on TCIA were used in this study: one for training the model (with internal validation) [18] (denoted as DS1) and another one for independent external validation of the model (denoted as DS2) [19].
Data for DS1 and DS2 were acquired on hybrid PET-CT scanners, and we considered only the pre-treatment PET images from these scanners. Data for DS1 were collected across four hospitals (HMR, HGJ, CHUS, CHUM) in Canada, and FDG was used as the radiopharmaceutical for PET imaging. For the HGJ cohort, the slice thickness resolution of FDG-PET (GE Healthcare-Discovery ST) was 3.27 mm for all patients and the median in-plane resolution was 3.52 × 5.52 mm2. For the CHUS cohort, the slice thickness resolution of FDG-PET (Philips-GeminiGXL) was 4 mm for all patients with a median in-plane resolution of 4 × 4 mm2. For the HMR cohort, the slice thickness resolution of FDG-PET (GE healthcare-Discovery STE) was 3.27 mm for all patients with a median in-plane resolution of 3.52 × 3.52 mm2, and for the CHUM cohort, the slice thickness resolution of FDG-PET (GE healthcare-Discovery STE) was 4 mm with a median in-plane resolution of 4 × 4 mm2. For the 102 patients included in DS2, PET images were obtained on GE Medical System’s Discovery RX, Discovery ST, Discovery HR and Discovery STE hybrid PET/CT by injecting 18F-labeled FDG intravenously. The slice thickness resolution for PET was 3.27 mm for all patients and the median in-plane resolution was 5.47 × 5.47 mm2. Further imaging protocol details of DS1 and DS2 can be found in [20,21].
The inclusion criteria for DS1 and DS2 were defined as follows. No cases demonstrated proven metastases at the time of presentation in both datasets. Cases included in the analysis had a follow-up period exceeding 5 months and specifically excluded cases where a pre-treatment PET scan was taken before or after 3 months of diagnosis (biopsy).
In this study, we modelled three outcomes with different follow-up periods: all-cause mortality (ACM) for 36 months, locoregional residual/recurrent disease (LR) for 24 months and distant metastasis (DM) for 24 months. Different follow-up periods for these outcomes were considered accounting for the available information in these two datasets. ACM was defined as cases where death occurred due to any cause from the time of imaging. LR was defined as the cases where cancer had remained or recurred in the same place as the original cancer, or the cancer had grown into lymph nodes or tissues near the original disease. DM was defined as cases where cancer had spread to organs or tissues far from the original cancer. These definitions closely align with the definitions provided in [21]. Patients alive/without LR/without DM at the time of analysis were censored at the date of the last follow-up.

2.2. Segmentation

Primary tumors on the PET images on both DS1 and DS2 were manually segmented by a radiologist specializing in head and neck imaging (JW) using LIFEx software (version 7.2.3) available at https://www.lifexsoft.org/ (accessed on 9 June 2022). The scanner setting in DS2 differed from DS1. LIFEx does not give feature values for tumors with <64 voxels as evidenced in the literature [22].

2.3. Preprocessing

Image preprocessing before feature extraction in both datasets was carried out on the LIFEx software (version 7.2.3). DS1 included data from multiple centers, but feature-level harmonization was not performed due to an insufficient number of samples in one of the cohorts. In PET imaging, if the neighboring voxels are not isotropic, it is necessary to resample them to an isotropic voxel size to make the feature extraction rotationally invariant [23]. The images were spatially resampled to a voxel size of 4 × 4 × 4 using a polynomial of degree 5 (quintic Lagrangian). Intensity discretization is an important preprocessing step to group close gray levels together before feature extraction to reduce the impact of noise [23]. In the context of PET imaging, absolute resampling (discretization) with 64 grey levels (bin size = 0.3125) between 0 and 20 standardized uptake value (SUV) units were used since the majority of SUV values in the training set (DS1) were between 0 and 20 SUV units. Radiomic features were extracted from primary tumors segmented on original images.

2.4. Feature Extraction

Hand-crafted features were extracted from the segmented image using LIFEx software. The software calculates a broad range of features in compliance with the Image Biomarker Standardization Initiative (IBSI). Radiomic features were extracted in a three-dimensional (3D) space. In total, 151 features across six feature classes (shape, first order, gray level co-occurrence matrix (GLCM), gray level run length matrix (GLRLM), neighborhood gray-tone difference matrix (NGTDM), and gray level size zone matrix (GLSZM)) were extracted from the PET images. After removing non-informative features (features with null values, features with similar values for all cases, feature values distributed in x, y, z axes, and feature values spread across different layers), 124 radiomic features were included in the final model development. The details of the radiomic features included in the analysis and the summary statistics of these features in DSI are provided in the Supplementary material (Tables S2 and S3). In addition to radiomics features, the model also included age, sex, stage of cancer and site of the primary tumor. The non-radiomics features were defined in the protocol details of DS1 and DS2 [20,21]. Due to limitations with sample sizes and to ensure that DS1 and DS2 have similar distributions of cancer sites and stages, some site and stage categories were combined.

2.5. Data Preparation

After the extraction of features, we developed a pipeline within the Python programming environment to facilitate the stages from data preparation to model development. The pipeline encompassed feature scaling, feature selection, model training, model evaluation and external validation. The pipeline predominantly employed Python (version 3.9.7) packages such as scikit-learn (version 0.24.2) [24] for data preparation and scikit-survival (version 0.16.0) [25] for developing, evaluating and validating survival models.
Since all segmentation tasks were conducted by a single operator, we did not conduct any segmentation reliability analysis. None of the features included in the analysis had any missing values. For the numeric radiomic features of DS1, feature scaling was performed using the min–max scaling technique. Categorical variables were transformed through one hot encoding. To reduce overfitting and to ensure robustness and generalizability of the developed models, we employed a robust feature selection strategy including hyperparameter optimization and cross validation, the details of which are provided in the following sections.
Feature selection was implemented by assessing the correlation coefficients between features—with highly correlated features excluded using a threshold of 0.95—which gave the best performance in a cardiac study [26]. Following the feature selection, we incorporated 65 radiomic features in the final model development stage. The correlation heatmaps for different classes of features are provided in the Supplementary Material (Figures S1–S6). An overview of the entire modelling pipeline is presented in Figure 1.

2.6. Model Training and Validation

The scikit-survival [25] Python module for survival analysis was used to model time-to-event outcomes (ACM, LR and DM). The four algorithms evaluated for fitting survival models on the data were: penalized Cox model, random forest, gradient boosted model, and support vector machine. Detailed descriptions of these models are available in the literature [27,28,29,30]. Observed survival estimates for all outcomes were presented using the Kaplan–Meier plots. The survival models considered the right-censored survival data with each time-to-event data containing a survival time (time between imaging and the event) and a status indicator (event observed or censored) [13].
Hyperparameter tuning was conducted using the randomized search cross-validation. The final model in the training set (DS1) was selected after five-fold cross-validation. To assess the contribution of each feature to the performance of the fitted model, permutation feature importance was estimated in the training set (DS1) by repeatedly shuffling each feature 1000 times [31]. The entire cohort of DS1 was used for model training. External validation of the model was performed using DS2 to assess the generalizability of the model.

2.7. Model Performance Evaluation

The model performances were evaluated using Harrell’s concordance index (C-index) to measure the discriminatory power and integrated Brier scores (IBS) to assess the accuracy of the predicted survival function and model calibration [32]. A C-index value of 1 signifies perfect discrimination and 0.5 indicates random prediction. IBS range from 0 to 1 represents perfect and worst discrimination and calibration, respectively. A model with a Brier score below 0.25 indicates a potentially useful model [32]. Permutation feature importance identifies the feature with the highest feature importance, indicating that the feature has the greatest impact on the model’s performance [31]. The model presenting the best performance metrics for each outcome based on the internal validation (i.e., following five-fold cross-validation) was selected as the final model. IBS does not apply to the support vector machine survival model since it does not provide an estimate of the survival function [33].

3. Results

3.1. Patient Characteristics

The characteristics of patients included in the study are summarized in Table 1. DS1 was a Canadian-based cohort with 232 patients (182 men and 50 women; mean age 63.03 ± 10.38 years), and DS2 was a US-based cohort with 102 patients (90 men and 12 women; mean age, 58.18 ± 8.96 years). In both DS1 and DS2, the major site of the tumor was the oropharynx: in DS1 71.55% and in DS2, 73.55%. The preferred treatment modality in both datasets was chemoradiotherapy with or without surgery. In DS1, 31 patients did not survive beyond 36 months, 25 patients developed locoregional recurrence or residual disease within 24 months, and 22 patients had distant metastasis within 24 months of follow-up.

3.2. Machine Learning-Based Prognostic Models

We considered a total of 65 prognostic radiomics and four clinical features for each time-to-event outcome (ACM, LR, DM) to train four models. Table 2 presents the performance metrics of all models in internal and external validation data for all outcome variables.

3.2.1. All-Cause Mortality

The penalized Cox model was selected as the best-performing model based on performance metrics to predict ACM. It demonstrated a reasonable performance in the validation set (DS2) with a C-index of 0.70 and an IBS of 0.12. Random forest and support vector machine also showed comparable performances (Table 2). The Kaplan–Meier curve in Figure 2 suggests that the survival probability was more than 0.85 at 36 months and displays a higher proportion of censoring than the number of events of interest during the follow-up period. The oropharynx as the site of primary tumor is the significant feature that contributes to the performance of the penalized Cox model alongside shape-based (maximum 3D diameter), GLCM_inverse variance and other clinical features (Figure 3). The hyperparameters for the penalized Cox model are provided in the Supplementary material (Table S4).

3.2.2. Locoregional Recurrent/Residual Disease

Following five-fold cross-validation and evaluation of performance metrics, the random forest survival model was identified as the best model to predict LR. It also demonstrated good performance in the validation set (DS2) with a C-index of 0.76 and an IBS of 0.07 (Table 2). The Kaplan–Meier curve suggests that the LR-free survival probability was more than 0.875 at 24 months (Figure 4) with an indication of a higher proportion of censoring during the study period as captured by the wider confidence interval around the survival estimates. Age, oropharynx as tumor site, GLCM-based features (clustershade and correlation) and first-order feature (IH_MaxHGrGL: Intensity Histogram Maximum Histogram Gradient Gray Level)) were the significant features contributing to the performance of the random forest model (Figure 5). The hyperparameters for the random forest are provided in the Supplementary material (Table S4).

3.2.3. Distant Metastasis

The penalized Cox model was selected as the best-performing model for distant metastasis (DM). It had a reasonably better performance in the validation set (DS2) with a C-index of 0.70 and an IBS of 0.08 (Table 2). The Kaplan–Meier curve suggests that DM-free survival probability was more than 0.90 at 24 months (Figure 6) displaying a wider confidence interval of the survival estimates later in the period due to an increased proportion of censoring events. Clinical features (site of the primary tumor, sex, stage of cancer) were the significant features contributing to the performance of the penalized Cox model in addition to shape-based compactness, GLSZM_GLNU (GLSZM Gray-Level Non-Uniformity) and NGTDM_Coarseness (Figure 7). The hyperparameters for the penalized Cox model are provided in the Supplementary material (Table S4).

4. Discussion

Employing a robust ML-based modelling approach combined with external model validation, our study highlights the potential utility of PET radiomics features in predicting adverse outcomes in patients with HNSCC. Our findings suggest that, for predicting ACM and DM in HNSCC using PET radiomics, the penalized Cox model performs better, while the random forest model exhibits favorable performance for LR. The time-to-event models hold particular significance within the context of HNSCC as the 5-year survival rate in this malignancy remains below 50% and prognosis is influenced by factors like disease stage, locoregional recurrence and distant metastasis [34]. From the PET radiomic features, information about tumor heterogeneity could be assessed, that expands on the information available from laboratory tests, clinical reports and genomic or proteomic assays [35]. Radiomic features, when used for machine learning modelling, macroscopically decode the phenotype of pathophysiological structures, providing valuable insights about disease diagnosis and prognosis, thereby suggesting its correlation with tumor biology [35,36]. For example, for the radiomic feature NGTDM_Coarseness that quantifies tumor granularity, a coarse texture indicates a high degree of local uniformity in intensity, whereas for NGTDM_Busyness, a busy texture implies rapid changes in intensity from one pixel to the neighboring pixel [37,38]. In head and neck cancer NGTDM coarseness and busyness can differentiate primary tumor and lymph nodes from healthy tissues [38]. The application of FDG-PET radiomics-based ML models may play a pivotal role in HNSCC management, assisting clinicians in predicting the recurrence risk and other adverse outcomes, and hence facilitating patient stratification for tailored treatments [39].
When compared with other models, we observed that the penalized Cox model performed reasonably well in predicting ACM and DM in independent datasets. These findings are consistent with previous HNSCC prognostic studies, where the Cox proportional hazard and the random forest models showed comparable performances in external validation for ACM [16]. There are, however, reports suggesting that the random forest survival model for DM prediction achieved better performance in external validation cohorts [16]. The penalized Cox model’s satisfactory performance for these outcomes could be due to its ability to enhance the predictive ability by effectively increasing the number of events per feature through the regularization technique of shrinking the regression parameters towards zero. The penalized Cox model proves particularly effective in high-dimensional survival data settings [40]. A study on oropharyngeal carcinoma reported that the penalized Cox model with radiomic features performed better in the external validation cohort for survival prediction [41].
We observed that the random forest model performed better in predicting LR. However, one study indicated that the Cox proportional hazard model had better performance in the external validation cohort for LR outcome prediction [16]. The superior performance of the random forest survival model could be attributed to its property of handling outliers and right-censored data, its ability to accommodate interactions between variables and non-linearity, and its non-dependence on the proportionality assumption [42,43].
All selected survival models reported in this study demonstrated acceptable estimates for concordance index (C-index) and integrated Brier score (IBS) both on training and validation datasets. Assessing the predictive performance of a survival model is important to determine the quality of the developed model [44]. The C-index is a widely used performance evaluation metric in the majority of the reported studies focusing on HNSCC PET [16]. The C-index is a valuable tool suggesting the performance of the model to discriminate between patients based on their risk. On the other hand, the Brier score is used to evaluate the overall performance of survival models, measuring both calibration and discrimination at a given time point. The IBS provides the average Brier score over a specified time period [45].
Our study, based on the permutation feature importance, suggested that clinical features in addition to shape-based and second-order features were the most important for the predictive model setting. Interestingly, we did not observe strong evidence of the importance of SUV-based features as reported [46]. This could be due to the methodology used to identify important features in ML models. Permutation feature importance, a model agnostic technique, reflects how important a feature is to the model rather than the predictive value of a feature to the outcome [31]. Notwithstanding, we noted similar features were reported to be predictive in other studies. Different HNSCC PET radiomic studies reported a range of features important for predicting different outcomes. Significant predictors for predicting ACM were reported as age, tumor site, T-stage, as well as shape-based and GLCM-based features. Recurrence prediction, on the other hand, was associated with age, tumor volume, and second-order radiomic features. Typical features for predicting DM are clinical variables [16]. It is worth noting that these studies primarily employed the Cox proportional hazards model with a limited number of features to identify key predictors of outcomes. Currently, there exist challenges to assess the statistical significance of predictive variables in non-linear ML models and to compare them with the inferential framework of the standard multivariable Cox survival model, which provides estimates of effect size and their statistical significance [47].

Limitations and Recommendations

While this study offers a comprehensive comparison of machine learning methods for HNSCC prognosis incorporating time-to-event data, it is not without limitations. Firstly, the retrospective nature of the study resulted in the exclusion of several clinical variables such as HPV (human papilloma virus) status and smoking status that could potentially serve as predictors of the outcome of interest, highlighting the need for a prospective study. Secondly, despite external validation of the calibrated model, the machine learning models were trained on a relatively small dataset without conducting an appropriate sample size calculation, as recommended for modelling studies [48]. The small sample size also hindered the development of site specific and stage specific HNSCC survival models which would otherwise provide added insights to the clinicians. Our study was constrained by the availability of publicly available resources. This study solely relied on radiomics features derived from pre-treatment PET images; however, adopting a multi-omics approach related to HNSCC could improve patient management demonstrating promising results [49,50]. Furthermore, implementing the Combat harmonization tool to address multicenter data variability and accounting for imaging protocol effects may further enhance the performance of the prognostic models [51,52]. Our study involved only a single radiologist for manual tumor segmentation. However, recognizing the possibility of inter-operator variability, it is recommended to include multiple radiologists for tumor segmentation to ensure reliability and reproducibility of segmentation results, by following a clear protocol [53]. Finally, we explored only a selected set of machine learning models to identify the best-performing model, while several other machine learning or deep learning algorithms for time-to-event analysis have been recently reported [54,55].

5. Conclusions

This study evaluated the performance metrics of four time-to-event machine learning algorithms for the prediction of ACM, LR and DM in HNSCC using multi-dimensional PET radiomics features, identified the best model and validated the model on an external dataset. For ACM and DM, the penalized Cox model, and for LR, the random forest model exhibited better performance compared to the other models. Further training and validations of the models in a larger cohort, adopting a multi-omics approach and inclusion of other clinical features, are required to facilitate personalized treatment planning in clinical practice.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers16122195/s1. Table S1: CLEAR checklist without explanations. Table S2: List of LIFEx (v7.2.3) features used for analysis. Table S3: Summary statistics of radiomic features in dataset 1 (training set). Table S4: Hyperparameters for the best model used for external validation for all outcomes. Figure S1: Correlation heatmap for age and shape-based features. Figure S2: Correlation heatmap for first-order features. Figure S3: Correlation heatmap for gray level co-occurrence matrix (GLCM) features. Figure S4: Correlation heatmap for gray level run length matrix (GLRLM) features. Figure S5: Correlation heatmap for neighborhood gray-tone difference matrix (NGTDM) features. Figure S6: Correlation heatmap for gray level size zone matrix (GLSZM) features.

Author Contributions

Conceptualization, M.M.P., F.M., A.W. and M.N.; Resources: M.M.P. and J.W.; Data curation: M.M.P. and J.W.; Methodology: M.M.P. and M.N.; Software: M.M.P., J.W. and M.N.; Formal analysis: M.M.P.; Investigation: M.M.P., J.W. and M.N.; Visualization: M.M.P. and M.N.; Validation: M.M.P. and M.N.; Supervision: F.M., A.W. and M.N.; Writing—original draft: M.M.P.; Writing—reviewing and editing: M.M.P., J.W., F.M., A.W. and M.N. All authors have read and agreed to the published version of the manuscript.

Funding

MMP acknowledges the funding by the University of Aberdeen under the Elphinstone Scholarship.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Primary PET image data used in this study are available to download in The Cancer Imaging Archive (TCIA) (https://www.cancerimagingarchive.net/) (accessed on 1 February 2023). The secondary feature data are available on request.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Bhatia, A.; Burtness, B. Treating Head and Neck Cancer in the Age of Immunotherapy: A 2023 Update. Drugs 2023, 83, 217–248. [Google Scholar] [CrossRef] [PubMed]
  2. Chow, Q.M.L. Head and Neck Cancer. N. Engl. J. Med. 2020, 382, 60–72. [Google Scholar] [CrossRef]
  3. Gong, Y.; Bao, L.; Xu, T.; Yi, X.; Chen, J.; Wang, S.; Pan, Z.; Huang, P.; Ge, M. The Tumor Ecosystem in Head and Neck Squamous Cell Carcinoma and Advances in Ecotherapy. Mol. Cancer 2023, 22, 68. [Google Scholar] [CrossRef]
  4. Chaves, P.; Garrido, M.; Oliver, J.; Pérez-Ruiz, E.; Barragan, I.; Rueda-Domínguez, A. Preclinical Models in Head and Neck Squamous Cell Carcinoma. Br. J. Cancer 2023, 128, 1819–1827. [Google Scholar] [CrossRef]
  5. Goel, B.; Tiwari, A.K.; Pandey, R.K.; Singh, A.P.; Kumar, S.; Sinha, A.; Jain, S.K.; Khattri, A. Therapeutic Approaches for the Treatment of Head and Neck Squamous Cell Carcinoma-An Update on Clinical Trials. Transl. Oncol. 2022, 21, 101426. [Google Scholar] [CrossRef] [PubMed]
  6. Lambin, P.; Zindler, J.; Vanneste, B.G.L.; Van De Voorde, L.; Eekers, D.; Compter, I.; Panth, K.M.; Peerlings, J.; Larue, R.T.H.M.; Deist, T.M.; et al. Decision Support Systems for Personalized and Participative Radiation Oncology. Adv. Drug Deliv. Rev. 2017, 109, 131–153. [Google Scholar] [CrossRef] [PubMed]
  7. Jung, J.-O.; Crnovrsanin, N.; Wirsik, N.M.; Nienhüser, H.; Peters, L.; Popp, F.; Schulze, A.; Wagner, M.; Müller-Stich, B.P.; Büchler, M.W.; et al. Machine Learning for Optimized Individual Survival Prediction in Resectable Upper Gastrointestinal Cancer. J. Cancer Res. Clin. Oncol. 2023, 149, 1691–1702. [Google Scholar] [CrossRef]
  8. Kourou, K.; Exarchos, T.P.; Exarchos, K.P.; Karamouzis, M.V.; Fotiadis, D.I. Machine Learning Applications in Cancer Prognosis and Prediction. Comput. Struct. Biotechnol. J. 2015, 13, 8–17. [Google Scholar] [CrossRef]
  9. Huynh, B.N.; Groendahl, A.R.; Tomic, O.; Liland, K.H.; Knudtsen, I.S.; Hoebers, F.; van Elmpt, W.; Malinen, E.; Dale, E.; Futsaether, C.M. Head and Neck Cancer Treatment Outcome Prediction: A Comparison between Machine Learning with Conventional Radiomics Features and Deep Learning Radiomics. Front. Med. 2023, 10, 1217037. [Google Scholar] [CrossRef]
  10. Hu, C.; Steingrimsson, J.A. Personalized Risk Prediction in Clinical Oncology Research: Applications and Practical Issues Using Survival Trees and Random Forests. J. Biopharm. Stat. 2018, 28, 333–349. [Google Scholar] [CrossRef]
  11. Tomaszewski, M.R.; Gillies, R.J. The Biological Meaning of Radiomic Features. Radiology 2021, 298, 505–516. [Google Scholar] [CrossRef]
  12. Palomino-Fernández, D.; Milara, E.; Galiana, Á.; Sánchez-Ortiz, M.; Seiffert, A.P.; Jiménez-Almonacid, J.; Gómez-Grande, A.; Ruiz-Solís, S.; Ruiz-Alonso, A.; Gómez, E.J.; et al. Textural and Conventional Pretherapeutic [18F]FDG PET/CT Parameters for Survival Outcome Prediction in Stage III and IV Oropharyngeal Cancer Patients. Appl. Sci. 2024, 14, 1454. [Google Scholar] [CrossRef]
  13. Yao, S.; Wang, X. Statistical and Machine Learning Methods for Discovering Prognostic Biomarkers for Survival Outcomes BT. In Statistical Genomics; Fridley, B., Wang, X., Eds.; Springer: New York, NY, USA, 2023; pp. 11–21. ISBN 978-1-0716-2986-4. [Google Scholar]
  14. Spooner, A.; Chen, E.; Sowmya, A.; Sachdev, P.; Kochan, N.A.; Trollor, J.; Brodaty, H. A Comparison of Machine Learning Methods for Survival Analysis of High-Dimensional Clinical Data for Dementia Prediction. Sci. Rep. 2020, 10, 20410. [Google Scholar] [CrossRef] [PubMed]
  15. Turkson, A.J.; Ayiah-Mensah, F.; Nimoh, V. Handling Censoring and Censored Data in Survival Analysis: A Standalone Systematic Literature Review. Int. J. Math. Math. Sci. 2021, 2021, 9307475. [Google Scholar] [CrossRef]
  16. Philip, M.M.; Welch, A.; McKiddie, F.; Nath, M. A Systematic Review and Meta-Analysis of Predictive and Prognostic Models for Outcome Prediction Using Positron Emission Tomography Radiomics in Head and Neck Squamous Cell Carcinoma Patients. Cancer Med. 2023, 12, 16181–16194. [Google Scholar] [CrossRef]
  17. Kocak, B.; Baessler, B.; Bakas, S.; Cuocolo, R.; Fedorov, A.; Maier-Hein, L.; Mercaldo, N.; Müller, H.; Orlhac, F.; Pinto dos Santos, D.; et al. CheckList for EvaluAtion of Radiomics Research (CLEAR): A Step-by-Step Reporting Guideline for Authors and Reviewers Endorsed by ESR and EuSoMII. Insights Imaging 2023, 14, 75. [Google Scholar] [CrossRef]
  18. Vallières, M. Data from Head-Neck-PET-CT—The Cancer Imaging Archive (TCIA). Available online: https://wiki.cancerimagingarchive.net/display/Public/Head-Neck-PET-CT (accessed on 1 February 2023).
  19. Grossberg, A.; Elhalawani, H.; Mohamed, A.; Mulder, S.; Williams, B.; White, A.L.; Zafereo, J.; Wong, A.J.; Berends, J.E.; AboHashem, S.; et al. HNSCC—The Cancer Imaging Archive (TCIA) Public Access—Cancer Imaging Archive Wiki. Available online: https://wiki.cancerimagingarchive.net/display/Public/HNSCC (accessed on 16 October 2023).
  20. Vallières, M.; Kay-Rivest, E.; Perrin, L.J.; Liem, X.; Furstoss, C.; Aerts, H.J.W.L.W.L.; Khaouam, N.; Nguyen-Tan, P.F.; Wang, C.-S.S.; Sultanem, K.; et al. Radiomics Strategies for Risk Assessment of Tumour Failure in Head-and-Neck Cancer. Sci. Rep. 2017, 7, 10117. [Google Scholar] [CrossRef]
  21. Grossberg, A.J.; Mohamed, A.S.R.; El Halawani, H.; Bennett, W.C.; Smith, K.E.; Nolan, T.S.; Williams, B.; Chamchod, S.; Heukelom, J.; Kantor, M.E.; et al. Data Descriptor: Imaging and Clinical Data Archive for Head and Neck Squamous Cell Carcinoma Patients Treated with Radiotherapy. Sci. Data 2018, 5, 180173. [Google Scholar] [CrossRef] [PubMed]
  22. Cook, G.J.R.; Siddique, M.; Taylor, B.P.; Yip, C.; Chicklore, S.; Goh, V. Radiomics in PET: Principles and Applications. Clin. Transl. Imaging 2014, 2, 269–276. [Google Scholar] [CrossRef]
  23. Orlhac, F.; Nioche, C.; Klyuzhin, I.; Rahmim, A.; Buvat, I. Radiomics in PET Imaging: A Practical Guide for Newcomers. PET Clin. 2021, 16, 597–612. [Google Scholar] [CrossRef]
  24. Scikit-Learn: Machine Learning in Python—Scikit-Learn 1.2.1 Documentation. Available online: https://scikit-learn.org/stable/ (accessed on 26 January 2023).
  25. Pölsterl, S. Scikit-Survival: A Library for Time-to-Event Analysis Built on Top of Scikit-Learn. J. Mach. Learn. Res. 2020, 21, 1–6. [Google Scholar]
  26. Lo Iacono, F.; Maragna, R.; Pontone, G.; Corino, V.D.A. A Robust Radiomic-Based Machine Learning Approach to Detect Cardiac Amyloidosis Using Cardiac Computed Tomography. Front. Radiol. 2023, 3, 1193046. [Google Scholar] [CrossRef]
  27. Ishwaran, H.; Kogalur, U.B.; Blackstone, E.H.; Lauer, M.S. Random Survival Forests. Ann. Appl. Stat. 2008, 2, 841–860. [Google Scholar] [CrossRef]
  28. Pölsterl, S.; Navab, N.; Katouzian, A. An Efficient Training Algorithm for Kernel Survival Support Vector Machines. arXiv 2016, arXiv:1611.07054. [Google Scholar]
  29. Hothorn, T.; Bühlmann, P.; Dudoit, S.; Molinaro, A.; Van Der Laan, M.J. Survival Ensembles. Biostatistics 2005, 7, 355–373. [Google Scholar] [CrossRef]
  30. Simon, N.; Friedman, J.; Hastie, T.; Tibshirani, R. Regularization Paths for Cox’s Proportional Hazards Model via Coordinate Descent. J. Stat. Softw. 2011, 39, 1–13. [Google Scholar] [CrossRef]
  31. 4.2. Permutation Feature Importance—Scikit-Learn 1.4.1 Documentation. Available online: https://scikit-learn.org/stable/modules/permutation_importance.html (accessed on 29 February 2024).
  32. Adeoye, J.; Hui, L.; Koohi-moghadam, M.; Yan, J.; Choi, S.; Thomson, P. Comparison of Time-to-Event Machine Learning Models in Predicting Oral Cavity Cancer Prognosis. Int. J. Med. Inform. 2022, 157, 104635. [Google Scholar] [CrossRef]
  33. Evaluating Survival Models—Scikit-Survival 0.22.2. Available online: https://scikit-survival.readthedocs.io/en/stable/user_guide/evaluating-survival-models.html (accessed on 23 February 2024).
  34. Pisani, P.; Airoldi, M.; Allais, A.; Aluffi Valletti, P.; Battista, M.; Benazzo, M.; Briatore, R.; Cacciola, S.; Cocuzza, S.; Colombo, A.; et al. Metastatic Disease in Head & Neck Oncology. Acta Otorhinolaryngol. Ital. 2020, 40, S1–S86. [Google Scholar]
  35. Pineiro-Fiel, M.; Moscoso, A.; Pubul, V.; Ruibal, A.; Silva-Rodriguez, J.; Aguiar, P.; Piñeiro-Fiel, M.; Moscoso, A.; Pubul, V.; Ruibal, Á.; et al. A Systematic Review of PET Textural Analysis and Radiomics in Cancer. Diagnostics 2021, 11, 380. [Google Scholar] [CrossRef] [PubMed]
  36. Avanzo, M.; Wei, L.; Stancanello, J.; Vallières, M.; Rao, A.; Morin, O.; Mattonen, S.A.; El Naqa, I. Machine and Deep Learning Methods for Radiomics. Med. Phys. 2020, 47, e185. [Google Scholar] [CrossRef] [PubMed]
  37. Amadasun, M.; King, R. Texural Features Corresponding to Texural Properties. IEEE Trans. Syst. Man. Cybern. 1989, 19, 1264–1274. [Google Scholar] [CrossRef]
  38. Cook, G.J.R.; Yip, C.; Siddique, M.; Goh, V.; Chicklore, S.; Roy, A.; Marsden, P.; Ahmad, S.; Landau, D. Are Pretreatment 18F-FDG PET Tumor Textural Features in Non-Small Cell Lung Cancer Associated with Response and Survival After Chemoradiotherapy? J. Nucl. Med. 2013, 54, 107375. [Google Scholar] [CrossRef]
  39. Noortman, W.A.; Aide, N.; Vriens, D.; Arkes, L.S.; Slump, C.H.; Boellaard, R.; Goeman, J.J.; Deroose, C.M.; Machiels, J.-P.; Licitra, L.F.; et al. Development and External Validation of a PET Radiomic Model for Prognostication of Head and Neck Cancer. Cancers 2023, 15, 2681. [Google Scholar] [CrossRef]
  40. Ambler, G.; Seaman, S.; Omar, R.Z. An Evaluation of Penalised Survival Methods for Developing Prognostic Models with Rare Events. Stat. Med. 2012, 31, 1150–1161. [Google Scholar] [CrossRef]
  41. Niyoteka, S.; Seban, R.-D.; Rouhi, R.; Scarsbrook, A.; Genestie, C.; Classe, M.; Carré, A.; Sun, R.; La Greca Saint-Esteven, A.; Chargari, C.; et al. A Common [18F]-FDG PET Radiomic Signature to Predict Survival in Patients with HPV-Induced Cancers. Eur. J. Nucl. Med. Mol. Imaging 2023, 50, 4010–4023. [Google Scholar] [CrossRef]
  42. Wang, Y.; Deng, Y.; Tan, Y.; Zhou, M.; Jiang, Y.; Liu, B. A Comparison of Random Survival Forest and Cox Regression for Prediction of Mortality in Patients with Hemorrhagic Stroke. BMC Med. Inform. Decis. Mak. 2023, 23, 215. [Google Scholar] [CrossRef]
  43. Wang, H.; Li, G. A Selective Review on Random Survival Forests for High Dimensional Data. Quant. Bio-Sci. 2017, 36, 85–96. [Google Scholar] [CrossRef]
  44. Chen, H.C.; Kodell, R.L.; Cheng, K.F.; Chen, J.J. Assessment of Performance of Survival Prediction Models for Cancer Prognosis. BMC Med. Res. Methodol. 2012, 12, 102. [Google Scholar] [CrossRef]
  45. Haider, H.; Hoehn, B.; Davis, S.; Greiner, R. Effective Ways to Build and Evaluate Individual Survival Distributions. J. Mach. Learn. Res. 2020, 21, 3289–3351. [Google Scholar]
  46. Romesser, P.B.; Lim, R.; Spratt, D.E.; Setton, J.; Riaz, N.; Lok, B.; Rao, S.; Sherman, E.J.; Schöder, H.; Lee, N.Y. The Relative Prognostic Utility of Standardized Uptake Value, Gross Tumor Volume, and Metabolic Tumor Volume in Oropharyngeal Cancer Patients Treated with Platinum Based Concurrent Chemoradiation with a Pre-Treatment [(18)F] Fluorodeoxyglucose Positron Em. Oral Oncol. 2014, 50, 802–808. [Google Scholar] [CrossRef]
  47. Sundrani, S.; Lu, J. Computing the Hazard Ratios Associated With Explanatory Variables Using Machine Learning Models of Survival Data. JCO Clin. Cancer Inform. 2021, 5, 364–378. [Google Scholar] [CrossRef]
  48. Rajput, D.; Wang, W.-J.; Chen, C.-C. Evaluation of a Decided Sample Size in Machine Learning Applications. BMC Bioinform. 2023, 24, 48. [Google Scholar] [CrossRef]
  49. Ribeiro, I.P.; Esteves, L.; Caramelo, F.; Carreira, I.M.; Melo, J.B. Integrated Multi-Omics Signature Predicts Survival in Head and Neck Cancer. Cells 2022, 11, 2536. [Google Scholar] [CrossRef]
  50. Marcu, L.G.; Marcu, D.C. Current Omics Trends in Personalised Head and Neck Cancer Chemoradiotherapy. J. Pers. Med. 2021, 11, 1094. [Google Scholar] [CrossRef]
  51. Wu, H.; Liu, X.; Peng, L.; Yang, Y.; Zhou, Z.; Du, D.; Xu, H.; Lv, W.; Lu, L. Optimal Batch Determination for Improved Harmonization and Prognostication of Multi-Center PET/CT Radiomics Feature in Head and Neck Cancer. Phys. Med. Biol. 2023, 68, 22. [Google Scholar] [CrossRef]
  52. Xu, H.; Abdallah, N.; Marion, J.-M.; Chauvet, P.; Tauber, C.; Carlier, T.; Lu, L.; Hatt, M. Radiomics Prognostic Analysis of PET/CT Images in a Multicenter Head and Neck Cancer Cohort: Investigating ComBat Strategies, Sub-Volume Characterization, and Automatic Segmentation. Eur. J. Nucl. Med. Mol. Imaging 2023, 50, 1720–1734. [Google Scholar] [CrossRef]
  53. Philip, M.M.; Watts, J.; Niki, S.; Moeini, M.; Musheb, M.; Mckiddie, F. Comparison of Semi-Automatic and Manual Segmentation Methods for Tumor Delineation on Head and Neck Squamous Cell Carcinoma (HNSCC) Positron Emission Tomography (PET) Images Comparison of Semi-Automatic and Manual Segmentation Methods for Tumor Deline. Phys. Med. Biol. 2024, 69, 9. [Google Scholar] [CrossRef]
  54. Wiegrebe, S.; Kopper, P.; Sonabend, R.; Bischl, B.; Bender, A. Deep Learning for Survival Analysis: A Review. Artif. Intell. Rev. 2024, 57, 65. [Google Scholar] [CrossRef]
  55. Wang, P.; Li, Y.; Reddy, C.K. Machine Learning for Survival Analysis: A Survey. ACM Comput. Surv. 2019, 51, 1–36. [Google Scholar] [CrossRef]
Cancers 16 02195 g001
Figure 1. Flow diagram depicting the data structure and machine learning pipeline for fitting the time-to-event survival models.
Figure 1. Flow diagram depicting the data structure and machine learning pipeline for fitting the time-to-event survival models.
Cancers 16 02195 g001
Cancers 16 02195 g002
Figure 2. Kaplan–Meier curve along with patient at risk and number of observed events (all-cause mortality) in patients with head and neck squamous cell carcinoma followed up for 36 months.
Figure 2. Kaplan–Meier curve along with patient at risk and number of observed events (all-cause mortality) in patients with head and neck squamous cell carcinoma followed up for 36 months.
Cancers 16 02195 g002
Cancers 16 02195 g003
Figure 3. Permutation feature importance plot depicting important predictors for penalized Cox model for all-cause mortality (ACM) outcome. IH: Intensity Histogram; GLCM: Gray Level Cooccurrence Matrix; GLSZM: Gray Level Size Zone Matrix; NGTDM: Neighborhood Gray Tone Difference Matrix; ZSEntropy: Zone Size Entropy; MinGL: Minimum Gray Level; QCOD: Quartile Coefficient of Dispersion; MinHGr: Minimum Histogram Gradient; RMS: Root Mean Square.
Figure 3. Permutation feature importance plot depicting important predictors for penalized Cox model for all-cause mortality (ACM) outcome. IH: Intensity Histogram; GLCM: Gray Level Cooccurrence Matrix; GLSZM: Gray Level Size Zone Matrix; NGTDM: Neighborhood Gray Tone Difference Matrix; ZSEntropy: Zone Size Entropy; MinGL: Minimum Gray Level; QCOD: Quartile Coefficient of Dispersion; MinHGr: Minimum Histogram Gradient; RMS: Root Mean Square.
Cancers 16 02195 g003
Cancers 16 02195 g004
Figure 4. Kaplan–Meier curve along with patient at risk and number of observed events (LR, locoregional recurrent/residual disease) in patients with head and neck squamous cell carcinoma followed up for 24 months.
Figure 4. Kaplan–Meier curve along with patient at risk and number of observed events (LR, locoregional recurrent/residual disease) in patients with head and neck squamous cell carcinoma followed up for 24 months.
Cancers 16 02195 g004
Cancers 16 02195 g005
Figure 5. Permutation feature importance plot depicting important predictors for random forest model for locoregional recurrent/residual disease outcome. IH: Intensity Histogram; GLCM: Gray Level Cooccurrence Matrix; GLSZM: Gray Level Size Zone Matrix; NGTDM: Neighborhood Gray Tone Difference Matrix; GLRLM: Gray Level Run Length Matrix; MaxHGrGL: Maximum Histogram Gradient Gray Level; QCOD: Quartile Coefficient of Dispersion; LGLZE: Low Gray Level Zone Emphasis; MinHGr: Minimum Histogram Gradient; LZHGLE: Large Zone High GrayLevel Emphasis; LRHGLE: Long Run High Gray Level Emphasis; LGLRE: Low Gray Level Run Emphasis.
Figure 5. Permutation feature importance plot depicting important predictors for random forest model for locoregional recurrent/residual disease outcome. IH: Intensity Histogram; GLCM: Gray Level Cooccurrence Matrix; GLSZM: Gray Level Size Zone Matrix; NGTDM: Neighborhood Gray Tone Difference Matrix; GLRLM: Gray Level Run Length Matrix; MaxHGrGL: Maximum Histogram Gradient Gray Level; QCOD: Quartile Coefficient of Dispersion; LGLZE: Low Gray Level Zone Emphasis; MinHGr: Minimum Histogram Gradient; LZHGLE: Large Zone High GrayLevel Emphasis; LRHGLE: Long Run High Gray Level Emphasis; LGLRE: Low Gray Level Run Emphasis.
Cancers 16 02195 g005
Cancers 16 02195 g006
Figure 6. Kaplan–Meier curve along with patient at risk and number of observed events (DM, distant metastasis) in patients with head and neck squamous cell carcinoma followed up for 24 months.
Figure 6. Kaplan–Meier curve along with patient at risk and number of observed events (DM, distant metastasis) in patients with head and neck squamous cell carcinoma followed up for 24 months.
Cancers 16 02195 g006
Cancers 16 02195 g007
Figure 7. Permutation feature importance plot depicting important predictors for penalized Cox model for distant metastasis outcome. IH: Intensity Histogram; GLCM: Gray Level Cooccurrence Matrix; GLSZM: Gray Level Size Zone Matrix; NGTDM: Neighborhood Gray Tone Difference Matrix; GLRLM: Gray Level Run Length Matrix; GLNU: Gray Level Non-Uniformity; AppVolume: Approximate Volume; RMS: Root Mean Square; SRE: Short Run Emphasis; QCOD: Quartile Coefficient of Dispersion; NGLNU: Normalized Gray Level Non Uniformity; SZE: Small Zone Emphasis.
Figure 7. Permutation feature importance plot depicting important predictors for penalized Cox model for distant metastasis outcome. IH: Intensity Histogram; GLCM: Gray Level Cooccurrence Matrix; GLSZM: Gray Level Size Zone Matrix; NGTDM: Neighborhood Gray Tone Difference Matrix; GLRLM: Gray Level Run Length Matrix; GLNU: Gray Level Non-Uniformity; AppVolume: Approximate Volume; RMS: Root Mean Square; SRE: Short Run Emphasis; QCOD: Quartile Coefficient of Dispersion; NGLNU: Normalized Gray Level Non Uniformity; SZE: Small Zone Emphasis.
Cancers 16 02195 g007
Table 1. Characteristics of patients in the training (Dataset 1) and external validation cohort (Dataset 2).
Table 1. Characteristics of patients in the training (Dataset 1) and external validation cohort (Dataset 2).
VariableSummaryEstimate (Dataset 1)Estimate (Dataset 2)
Number of patientsTotal232102
SexMale18290
Female5012
Age (years)Overall: Mean (SD)63.03 (10.38)58.18 (8.96)
Primary site [n (%)]Oropharynx 166 (71.55)75 (73.53)
Nasopharynx22 (9.48)4 (3.92)
Hypopharynx8 (3.44)6 (5.88)
Others (Larynx, Glottis, Oral cavity, Sinus)36 (15.52)17 (16.67)
Stage of cancer [n (%)]Stage I/II (II, IIB)24 (10.34)4 (3.92)
Stage III55 (23.71)16 (15.69)
Stage IV (IV, IVA, IVB)153 (65.95)82 (80.39)
Treatment strategy [n (%)]Chemoradiation with/without surgery 196 (84.48)57 (55.88)
Chemoradiation with/without surgery and targeted therapy-7 (6.86)
Radiation with/without surgery36 (15.52)25 (24.51)
Radiation with/without surgery and targeted therapy-13 (12.75)
Number [n (%)]All-cause deaths31 (13.36)21 (20.59)
Locoregional recurrent/residual disease25 (10.78)14 (13.73)
Distant metastasis22 (9.48)12 (11.76)
Follow-up time in months [Median (Interquartile range)] All-cause mortality (ACM)41 (25.96)63 (48.99)
Locoregional recurrent/residual disease (LR) 38 (26.38)53 (60.31)
Distant metastasis (DM) 39 (27.43)53 (60.31)
Table 2. Performance comparison of various prognostic models in the training cohort (DS1) and external validation cohort (DS2).
Table 2. Performance comparison of various prognostic models in the training cohort (DS1) and external validation cohort (DS2).
Outcome of Interest ModelConcordance Index (CI)Integrated Brier Score (IBS)
Internal Validation (Dataset 1)External Validation (Dataset 2)Internal Validation (Dataset 1)External Validation (Dataset 2)
All-cause mortality (ACM)Penalized Cox0.780.700.050.12
Random forest 0.720.700.050.12
Gradient boosted model0.780.660.060.12
Support vector machine0.810.70NANA
Locoregional recurrent/residual disease (LR)Penalized Cox0.840.660.050.07
Random forest0.860.760.050.07
Gradient boosted model0.790.610.060.07
Support vector machine0.820.65NANA
Distant metastasis (DM)Penalized Cox0.790.700.040.08
Random forest0.760.700.040.08
Gradient boosted model0.790.710.040.09
Support vector machine0.770.71NANA
NA = Not available.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Philip, M.M.; Watts, J.; McKiddie, F.; Welch, A.; Nath, M. Development and Validation of Prognostic Models Using Radiomic Features from Pre-Treatment Positron Emission Tomography (PET) Images in Head and Neck Squamous Cell Carcinoma (HNSCC) Patients. Cancers 2024, 16, 2195. https://doi.org/10.3390/cancers16122195

AMA Style

Philip MM, Watts J, McKiddie F, Welch A, Nath M. Development and Validation of Prognostic Models Using Radiomic Features from Pre-Treatment Positron Emission Tomography (PET) Images in Head and Neck Squamous Cell Carcinoma (HNSCC) Patients. Cancers. 2024; 16(12):2195. https://doi.org/10.3390/cancers16122195

Chicago/Turabian Style

Philip, Mahima Merin, Jessica Watts, Fergus McKiddie, Andy Welch, and Mintu Nath. 2024. "Development and Validation of Prognostic Models Using Radiomic Features from Pre-Treatment Positron Emission Tomography (PET) Images in Head and Neck Squamous Cell Carcinoma (HNSCC) Patients" Cancers 16, no. 12: 2195. https://doi.org/10.3390/cancers16122195

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop