Development and Validation of a Prediction Model for Positive Findings of Preoperative Flexible Bronchoscopy in Patients with Peripheral Lung Cancer

Round 1

Reviewer 1 Report

Development and Validation of a Prediction Model for Positive Findings of Preoperative Flexible Bronchoscopy in Patients with Peripheral Lung Cancer

 

 

Abstract

(1) Background: It has yet to be determined whether preoperative flexible bronchoscopy (FB) should be routinely performed in patients with peripheral lung cancer. The aim of this study was to construct a model to predict the probability of positive FB findings, which would help assess the necessity of preoperative FB. (2) Methods: A total of 380 consecutive patients with peripheral lung cancer who underwent preoperative FB were recruited for this study. A prediction model was developed through a univariate and multivariate logistic regression with predictors including gender, age, body mass index (BMI), smoking, history of chronic lung diseases, respiratory symptoms, lesion size, lesion type, lesion location in bronchi, and lesion location in lobe. The predictive performance of the model was evaluated by validation using 1000 iterations of bootstrap resampling. Model discrimination was assessed using the area under the receiver operating characteristics curve (AUC), and calibration was assessed using the Brier score and calibration plots. (3) Results: The model suggested that male patients with respiratory symptoms, decreased BMI, solid lesions, and lesions located in lower-order bronchi were more likely to have positive FB findings. The AUC and Brier score of the model in internal validation was 0.784 and 0.162, respectively. The calibration curve for the probability of positive FB findings showed convincing concordance between the predicted and actual results. (4) Conclusions: A model to predict the probability of positive FB findings in patients with peripheral lung cancers was developed and validated. The model will facilitate clinicians in assessing the necessity of preoperative FB in patients with peripheral lung cancer.

Comments: The abstract is well constructed, however, the given Conclusions do not link well/cohesive with the aim / method

 

Furthermore, 3.3. should be included as part of the Method section, rather than as Results

3.3. Development and Validation of the Model 169

Univariate logistic regression was used to select variables including gender, age, 170 BMI, smoking, respiratory symptoms, lesion size, lesion type, and lesion location in bron-171 chi (all P values <0.1) that were associated with positive FB findings (Table 2). These var-172 iables were then progressed into the multivariate analysis. Based on the Akaike infor-173 mation criterion (AIC) achieved via “stepwise backward selection”, gender, BMI, respira-174 tory symptoms, lesion size, lesion type, and lesion location in bronchi were selected to 175 construct a model to predict the probability of positive FB findings in patients with pe-176 ripheral lung cancer (Table 3). A nomogram was established according to the model (Fig-177 ure 2). Each predictive variable received a corresponding score on the highest scoring line 178 of the nomogram through a vertical line. The scores of all variables were summed to ob-179 tain a total score. The probability corresponding to the total score is the predicted proba-180 bility of a positive preoperative FB finding.

 

 

5. Conclusions 300

In conclusion, an effective model to predict the probability of positive FB findings for 301 the patients with peripheral lung cancers was developed and validated. This model will 302 help clinicians assess the necessity of preoperative FB and make an individual recommen-303 dation for patients.

Comment: the Conclusions do not link well/cohesive with the aim / method

 

METHODS

Based on example stated below, the sample size of 380 used by this study might not able to produce a robust model, pls justify

 

Preoperative Airway Assessment Predictive Value of a Multivariate Risk Index

https://journals.lww.com/anesthesia-analgesia/Fulltext/1996/06000/Preoperative_Airway_Assessment__Predictive_Value.17.aspx

 

 

Using readily available and objective airway risk criteria, a multivariate model for stratifying risk of difficult endotracheal intubation was developed and its accuracy compared to currently applied clinical methods.We studied 10,507 consecutive patients who were prospectively assessed prior to general anesthesia with respect to mouth opening, thyromental distance, oropharyngeal (Mallampati) classification, neck movement, ability to prognath, body weight, and history of difficult tracheal intubation.

Author Response

On behalf of my co-authors, thank you very much for giving us the opportunity to revise our manuscript. We gratefully thank the editor and all reviewers for their time spent making their constructive remarks and useful suggestions. Those comments are all valuable and very helpful for revising and improving our paper, and of great importance to our research. We have studied comments carefully and have made revisions which we hope meet with approval. Below the comments of the reviewers are responded point by point and the revisions are indicated.

 

Comments 1: The abstract is well constructed, however, the given Conclusions do not link well/cohesive with the aim / method

Response 1:

Thank you for your suggestions. We have revised the article according to your suggestions as follows:

“Conclusions: Our prediction model estimated the pretest probability of positive FB findings in patients with peripheral lung cancers. Male, lower BMI, the presence of respiratory symptoms, larger lesions, solid lesions, and lesions located in lower-order bronchi were associated with increased positive FB findings. Use of our model can be of assistance when making clinical decisions about preoperative FB.”

You may see these details in lines 27–31 of the manuscript.

 

Comments 2: Furthermore, 3.3. should be included as part of the Method section, rather than as Results

Response 2:

Thank you for your suggestions. We have revised the article according to your suggestions as follows:

“According to the results of our analysis (Table 2, Table 3), BMI (OR, 0.924; 95% CI, 0.854–0.996; P value = 0.043), respiratory symptoms (P value = 0.016), lesion size (OR, 3.212; 95% CI, 1.869–5.560; P value <0.001), and lesion location in bronchi (OR, 0.248; 95% CI, 0.135–0.438; P value <0.001) are independent predictor factors for positive FB findings in patients with peripheral lung cancer. Based on the Akaike information criterion (AIC) achieved via stepwise backward selection, the gender and lesion type also were entered into the final model. Gender, BMI, respiratory symptoms, lesion size, lesion type, and lesion location in bronchi were chosen to construct a model to predict the probability of positive FB findings in patients with peripheral lung cancer (Table 3). A nomogram was established according to the model (Figure 2).”

You may see these details in lines 179–188 of the manuscript.

 

Comments 3: the Conclusions do not link well/cohesive with the aim / method

Response 3:

Thank you for your suggestions. We have revised the article according to your suggestions as follows:

“This study developed a parsimonious clinical prediction model for calculating the probability of positive FB findings in patients with peripheral lung cancers. Male, lower BMI, the presence of respiratory symptoms, larger lesions, solid lesions, and lesions located in lower-order bronchi were associated with increased positive FB findings. The model was validated and proved to have good discrimination. The model has the potential to assist physicians in making clinical decisions about preoperative FB.”

You may see these details in lines 335–340 of the manuscript.

 

Comments 4: Based on example stated below, the sample size of 380 used by this study might not able to produce a robust model, pls justify

 

Preoperative Airway Assessment Predictive Value of a Multivariate Risk Indexhttps://journals.lww.com/anesthesiaanalgesia/Fulltext/1996/06000/Preoperative_Airway_ Assessment__Predictive_Value.17.aspx

Using readily available and objective airway risk criteria, a multivariate model for stratifying risk of difficult endotracheal intubation was developed and its accuracy compared to currently applied clinical methods.We studied 10,507 consecutive patients who were prospectively assessed prior to general anesthesia with respect to mouth opening, thyromental distance, oropharyngeal (Mallampati) classification, neck movement, ability to prognath, body weight, and history of difficult tracheal intubation.

 

Response 4:

Thank you for your suggestions.

Peduzzi et al.[1] explored the minimum number of events per variable (EPV) required in multivariate analysis in their study. Through the data simulation experiment, they found that with the decrease of EPV, the distribution of regression coefficients gradually became scattered, and no longer converged to the real regression coefficients in the original data, especially when EPV < 10. When EVP ≥ 10, the regression coefficient can converge to the real regression coefficient, which indicates that the calculated regression model is more accurate. On this basis, they concluded that: for EPV values of 10 or greater, no major problems occurred. for EPV values less than 10, however, the regression coefficients were biased in both positive and negative directions; the large sample variance estimates from the logistic model both overestimated and underestimated the sample variance of the regression coefficients; the 90% confidence limits about the estimated values did not have proper coverage. Of course, EPV = 10 is just a rough Rule of Thumb. Some authors think that this principle can be relaxed under certain circumstances, such as EPV < 10 [2], and some authors think that EPV = 15 or 20 will be safer [3].

There were 114 patients with positive FB findings of the 380 patients included in our study, and we included 8 independent variables in multivariate regression analysis. Therefore, the EPV of this study is 14.25 (114/8=14.25), which meets the minimum requirement of 10 and is close to 15, showing that the sample size may be sufficient. However, we do take your suggestion seriously and plan to collect more samples in future studies.

 

References

1. Peduzzi, P.; Concato, J.; Kemper, E.; Holford, T.R.; Feinstein, A.R. A Simulation Study of the Number of Events per Variable in Logistic Regression Analysis. Journal of Clinical Epidemiology 1996, 49, 1373–1379, doi:10.1016/S0895-4356(96)00236-3.
2. Vittinghoff, E.; McCulloch, C.E. Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression. American Journal of Epidemiology 2007, 165, 710–718, doi:10.1093/aje/kwk052.
3. Frank E. Harrell, Regression Modeling Strategies, 2015

Reviewer 2 Report

Dear authors,

Peripheral lung cancer is becoming more and more common as screening programs, as well as early detection become possible, thus pre-operative bronchoscopy is an issue. I read your manuscript with interest.

Overall: Good use of English language. Interesting and well-presented.

Abstract: Well written. All info needed. Within word limit.

Introduction: Well written.

Methods: 

-Please define peripheral lesion (outer 1/3?)

-Were patients with lymphadenopathy excluded on the basis that they were inoperable or that they would have bronchoscopy otherwise for EBUS and the study wanted to focus on the group of patients that could skip bronchoscopy or something else? If skipping the bronchoscopy is the aim, all patients with lesions over 3 cm would also have to be examined with EBUS prior to operation for N staging, so would they also have to be excluded from the study group? (https://pubmed.ncbi.nlm.nih.gov/26034128/)

Results: Figure 2: risk of hypertension? Please correct or explain.

Discussion:

-If you find it suitable to comment: Using your prediction model how many false negative cases would be missed in the specific population? Although the aim of the study is the identification of a prediction model, it would be interesting to comment on: for those cases did the bronchoscopic findings change staging ex T2 for main bronchus involvement or T4 status for main carina involvement and did the different staging affect the operability? So is it safe to recommend not performing bronchoscopy?

-Could correlation between heavier smoking history and more positive bronchoscopic findings be partially explained by the increased posibility of developping a squamous carcinoma? Are there any ideas why BMI is a predictor (ex lower BMI correlated with cancer cachexia)?

Best regards.

Author Response

We would like to express our sincere gratitude to the reviewers and editors for their professional comments and suggestions, which has significantly raised the quality of the manuscript and has enable us to improve the manuscript. We have studied the comments carefully and have made revisions which we hope meet with approval. Below the comments of the reviewers are responded point by point and the revisions are indicated.

 

Methods:

Comment 1: Please define peripheral lesion (outer 1/3?)

Response 1:

Thank you so much for your suggestions. There is no uniform definition of peripheral lung cancer in the guidelines. The European Society of Thoracic Surgeons[1] and the National Comprehensive Cancer Network[2] define peripheral lung cancer as tumors within the outer one-third of the lung, whereas the American College of Chest Physicians[3] suggests that peripheral lung cancer is within the outer two-thirds. In addition, there are studies to define peripheral lung cancer defined as tumors arising from subsegmental or other distal bronchi and bronchioli[4]. Based on these definitions and the nature of our study, we defined peripheral lesions as tumors limited to subsegmental or other distal bronchi and bronchiole.

You may see these details in lines 104–105 of the manuscript.

 

Comment 2: Were patients with lymphadenopathy excluded on the basis that they were inoperable or that they would have bronchoscopy otherwise for EBUS and the study wanted to focus on the group of patients that could skip bronchoscopy or something else? If skipping the bronchoscopy is the aim, all patients with lesions over 3 cm would also have to be examined with EBUS prior to operation for N staging, so would they also have to be excluded from the study group? (https://pubmed.ncbi.nlm.nih.gov/26034128/)

Response 2:

Thank you for your suggestion. Yes, we excluded patients with lymphadenopathy for the reasons you described. As for patients with lesions > 3 cm, we would like to explain this point for you as follows: In China, many grassroots hospitals cannot perform invasive mediastinal lymph node staging according to the guidelines in a standardized manner due to the lack of medical equipment or differences in medical care. In fact, except for some patients highly suspected of N2/N3 lymph node involvement, most patients with lesions >3cm and cN0 on CT/PET-CT did not receive invasive mediastinal lymph node examination preoperatively. In addition, the Chinese Society of Clinical Oncology (CSCO) guidelines on the diagnosis and treatment of non-small cell lung cancer [5]do not provide clear suggestions on the conditions for invasive mediastinal lymph node staging, which may also be an important reason. Based on this practical clinical situation, we did not exclude patients with lesions >3cm in the study, for the reason that these patients may benefit from the study results in clinical practice. However, we do take your suggestion seriously and plan to conduct further research in the future.

 

Results:

Comment 3: Results: Figure 2: risk of hypertension? Please correct or explain.

Response 3:

We are very sorry! This is an error due to our negligence. The correct description should be: the risk of a positive FB findings. We have revised it. Thank you so much for your correction.

You may see these details in Figure 2(line 221) of the manuscript.

 

Discussion:

Comment 4: If you find it suitable to comment: Using your prediction model how many false negative cases would be missed in the specific population? Although the aim of the study is the identification of a prediction model, it would be interesting to comment on: for those cases did the bronchoscopic findings change staging ex T2 for main bronchus involvement or T4 status for main carina involvement and did the different staging affect the operability? So is it safe to recommend not performing bronchoscopy?

Response 4:

Thanks for your suggestion. Through statistical analysis, we can calculate that the false negative rate of the model is 35.1% (indicating that for all positive FB patients, the probability of being predicted as negative by the model is 35.1% ) when we take the predicted probability of 34.2% corresponding to the maximum Youden index as the cutoff value, while the negative predictive value (NPV) of the model is 84.6% (indicating that among the patients predicted to be negative by the model, the probability of being really negative is 84.6%). In order to reduce the possibility of missing patients with positive FB findings, clinicians can adjust the diagnostic threshold smaller to improve the sensitivity. But this is done at the expense of model specificity.

In addition, although we have determined the best cutoff value according to the maximum Youden index, different clinicians may flexibly choose different cutoff values according to clinical needs. For patients who are more likely to be bronchus involvement or main carina involvement according to clinical factors (such as obvious cough, or abnormal images of main airway found by CT, etc.), clinicians can appropriately reduce the diagnostic threshold, because the damage caused by possible false negative findings is obviously higher than the benefit brought by forgoing FB due to possible true negative findings. Of course, our model cannot cover all situations, but the use of this model can provide a quantifiable risk assessment for clinicians in uncertainty as an important consideration standard for decision-making.

 

Comment 5: Could correlation between heavier smoking history and more positive bronchoscopic findings be partially explained by the increased posibility of developping a squamous carcinoma? Are there any ideas why BMI is a predictor (ex lower BMI correlated with cancer cachexia)?

Response 5:

Thank you for your suggestions. We think your consideration is reasonable. Smoking history was associated with positive FB findings in univariate analysis (P=0.008). The possible explanation for this is that the incidence of squamous cell carcinoma is higher in smokers[6,7], and squamous cell carcinoma is more likely to be detected because it is closer to the central airway[7,8]. The results of our statistical analysis in this study confirm this hypothesis (Table1, Table2). However, smoking history was not associated with positive FB findings in multivariate analysis (P=0.271). The possible explanation is that men are more likely to get positive FB findings because larger airway diameter compared with women[9], and men have higher prevalence of smoking than women at the same time[6]. The correlation between smoking and male, squamous cell carcinoma may be the reason for its association with positive FB findings in univariate analysis. However, smoking history was not associated with positive FB findings in multivariate analysis, which also indicates that smoking history may not be the direct cause of positive FB findings.

You may see Tables in the attachment please.

Our study found that patients with lower BMI were more likely to get positive FB findings, which is an interesting finding. In this regard, we have guessed the following possible reasons: 1. Lower BMI may reflect the consumption of tumors on the body, indicating the activity of tumor growth. And the actively growing tumor may invade surrounding tissues more obviously; 2. According to clinical experience, obese patients with high BMI have relatively poor tolerance and cooperation to FB examination, which may affect the examination results to some extent. It is reported recently that patients with higher BMI showed a statistically significant increase in anxiety score (P=0.041)[10]. And higher anxiety may affect the patient's tolerance and cooperation with the examination, which confirms our thought to some extent. Of course, this result is likely to be caused by multiple reasons rather than one single reason, and we will further clarify it in future research.

 

References:

1. De Leyn, P.; Dooms, C.; Kuzdzal, J.; Lardinois, D.; Passlick, B.; Rami-Porta, R.; Turna, A.; Schil, P.V.; Venuta, F.; Waller, D.; et al. Revised ESTS Guidelines for Preoperative Mediastinal Lymph Node Staging for Non-Small-Cell Lung Cancer. European Journal of Cardio-Thoracic Surgery2014, 45, 787–798, doi:10.1093/ejcts/ezu028.
2. NCCN Guidelines: Non-Small Cell Lung Cancer, Version 2. 2022.Available at: http://www.nccn.org/.
3. Silvestri, G.A.; Gonzalez, A.V.; Jantz, M.A.; Margolis, M.L.; Gould, M.K.; Tanoue, L.T.; Harris, L.J.; Detterbeck, F.C. Methods for Staging Non-Small Cell Lung Cancer. Chest 2013, 143, e211S-e250S, doi:10.1378/chest.12-2355.
4. Shimosato Y, Hashimoto T, Kodama T, et al. Prognostic implications of fibrotic focus (scar) in small peripheral lung cancers. Am J Surg Pathol 1980;4:365–73.
5. Guidelines of Chinese society of clinical oncology (CSCO)NON-SMALL CELL LUNG CANCER 2022. Available at: http://www.csco.org.cn/cn/index.aspx.
6. Lin, H.-H.; Murray, M.; Cohen, T.; Colijn, C.; Ezzati, M. Effects of Smoking and Solid-Fuel Use on COPD, Lung Cancer, and Tuberculosis in China: A Time-Based, Multiple Risk Factor, Modelling Study. The Lancet 2008, 372, 1473–1483, doi:10.1016/S0140-6736(08)61345-8.
7. Gao, L.; Asmitanand, T.; Ren, H.; Wu, F.; Zhang, Y.; Li, X.; Di, L.; Song, Z.; Yang, T.; Chen, T.; et al. Fiber-Optic Bronchoscope and Detection of Lung Cancer: A Five Year Study. neo 2012, 59, 201–206, doi:10.4149/neo_2012_026.
8. Sereno, M.; Esteban, I.R.; Zambrana, F.; Merino, M.; Gómez-Raposo, C.; López-Gómez, M.; Sáenz, E.C. Squamous-Cell Carcinoma of the Lungs: Is It Really so Different? Critical Reviews in Oncology/Hematology 2012, 84, 327–339, doi:10.1016/j.critrevonc.2012.06.009.
9. Olivier, P.; Hayon-Sonsino, D.; Convard, J.P.; Laloë, P.-A.; Fischler, M. Measurement of Left Mainstem Bronchus Using Multiplane CT Reconstructions and Relationship Between Patient Characteristics or Tracheal Diameters and Left Bronchial Diameters. Chest 2006, 130, 101–107, doi:10.1378/chest.130.1.101.
10. Aljohaney, A. Level and Predictors of Anxiety in Patients Undergoing Diagnostic Bronchoscopy. Ann Thorac Med 2019, 14, 198, doi:10.4103/atm.ATM_38_19.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments are well addressed

Reviewer 2 Report

Dear authors,

I find the revised manuscript sufficiently improved. 

Best regards.