Effects of Antibody Response after Booster Vaccination on SARS-CoV-2 Breakthrough Infections and Disease Outcomes in Advanced Cancer Patients: A Prospective Analysis of the Vax-on-Third Study

Round 1

Reviewer 1 Report

The study describes the antibody response after triple vaccination to COVID-19 and its relation to break trough infections and the outcome of advanced cancer patients. The introduction adequately describes the background of the study. Methods are fully described, including all ethical and statistical issues. 

The results are presented clearly and concisely. The discussion fully addresses the results in terms of published studies. The authors are aware of the shortcomings of the study. However, if the authors did not include additional booster vaccine doses in the analysis, it could strongly influence the results. I would recommend that if the booster doses were received it should be included in the analysis.

Author Response

Answers to Comments and Suggestions for Authors: Reviewer 1

The study describes the antibody response after triple vaccination to COVID-19 and its relation to break trough infections and the outcome of advanced cancer patients. The introduction adequately describes the background of the study. Methods are fully described, including all ethical and statistical issues.

The results are presented clearly and concisely. The discussion fully addresses the results in terms of published studies. The authors are aware of the shortcomings of the study. However, if the authors did not include additional booster vaccine doses in the analysis, it could strongly influence the results. I would recommend that if the booster doses were received it should be included in the analysis.

- Accordingly, our multivariate analysis now includes among the clinical covariates the receipt of additional boosters, such as the fourth dose of tozinameran or bivalent vaccination, which were implemented as of March 2022. Since at that time 32% of the enrolled patients had already developed COVID-19 breakthrough infections, these pharmacological interventions cannot be considered applicable to the general study population. Evidence that additional booster doses may increase COVID-19 protection for cancer patients, even those who do not respond to the initial vaccine series, introduces potential confounding and bias that we could not foresee at the beginning of this research [Thakkar A, et al. Study of efficacy and longevity of immune response to third and fourth doses of COVID-19 vaccines in patients with cancer: A single arm clinical trial. Elife . 2023 Mar 28;12:e83694. doi: 10.7554/eLife.83694].  We have therefore correspondingly amended subsection 3.2 and the discussion section.

Reviewer 2 Report

This paper is very clearly presented and well-written. Among patients on active treatment for

advanced disease, the authors demonstrate that humoral activity was associated with the odds

of a breakthrough COVID-19 infection, and that the occurrence of a breakthrough infection was

associated with a shorter time to treatment failure. I have the following suggestions:

● Section 3.2: Please clarify in the results section how the IgG titer cut-point of 803

BAU/mL was arrived at (e.g. via the Youden index).

● Lines 170-171: The results presented in Table 2 are a logistic regression analysis. As such,

impaired humoral responses should be reported as associated with an increased “odds”

of breakthrough infection. See (e.g.):

○ Ranganathan P, Aggarwal R, Pramesh CS. Common pitfalls in statistical analysis:

Odds versus risk. Perspect Clin Res. 2015 Oct-Dec;6(4):222-4. doi:

10.4103/2229-3485.167092.

○ “Risk” is subsequently used several more times when “odds” would be more

appropriate.

● Section 3.3: Please clarify whether death from any cause was regarded as a treatment

failure event (as opposed to censoring).

● Section 3.3: In addition to the univariate hazard ratio, please consider reporting the

restricted mean time-to-treatment failure across the 18 months of follow-up for each

covariate. See (e.g.):

○ Uno H, Claggett B, Tian L, et al. Moving beyond the hazard ratio in quantifying

the between-group difference in survival analysis. JCO. 2014; 32(22):2380-2385.

○ McCaw ZR, Tian L, Wei J, et al. Choosing clinically interpretable summary

measures and robust analytic procedures for quantifying the treatment

difference in comparative clinical studies. Stat Med. 2021; 40(28):6235-6242. doi:

10.1002/sim.8971.

○ The proposed analysis can easily be conducted using the survRM2 R package:

https://cran.r-project.org/web/packages/survRM2/index.html.

● Line 243: Please cite the reference rate of breakthrough infection in the relevant

population (not only that the rate was higher in this study).

● Line 293: Was there any evidence of heterogeneity in the association between antibody

response and odds of breakthrough infection, or between breakthrough infection and

time to treatment failure, across the different types of cancer?

Author Response

Answers to Comments and Suggestions for Authors: Reviewer 2

This paper is very clearly presented and well-written. Among patients on active treatment for advanced disease, the authors demonstrate that humoral activity was associated with the odds of a breakthrough COVID-19 infection, and that the occurrence of a breakthrough infection was associated with a shorter time to treatment failure. I have the following suggestions:

• Section 3.2: Please clarify in the results section how the IgG titer cut-point of 803 BAU/mL was arrived at (e.g. via the Youden index).

- As suggested, we specified the use of the Youden index to identify the optimal IgG titer cut-point.

• Lines 170-171: The results presented in Table 2 are a logistic regression analysis. As such, impaired humoral responses should be reported as associated with an increased “odds” of breakthrough infection. See (e.g.): Ranganathan P, Aggarwal R, Pramesh CS. Common pitfalls in statistical analysis: Odds versus risk. Perspect Clin Res. 2015 Oct-Dec;6(4):222-4. doi:10.4103/2229-3485.167092.

- As the Reviewer correctly suggested, we revised Table 2 to check the significance of the "odds" of breakthrough infections in relation to different covariates.  In detail, a high antibody response level is now associated with significantly reduced odds [OR 0.004 (95% CI 0.001 – 0.019), p<0.001] clearly indicating a decreased likelihood of breakthrough infections.

○ “Risk” is subsequently used several more times when “odds” would be more appropriate.

-  We replaced the term "risk" with " odds" where the latter would be more appropriate.

• Section 3.3: Please clarify whether death from any cause was regarded as a treatment failure event (as opposed to censoring).

- We clarified that death without evidence of disease progression was considered a treatment failure event.

• Section 3.3: In addition to the univariate hazard ratio, please consider reporting the restricted mean time-to-treatment failure across the 18 months of follow-up for each covariate. See (e.g.):

○ Uno H, Claggett B, Tian L, et al. Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis. JCO. 2014; 32(22):2380-2385.

○ McCaw ZR, Tian L, Wei J, et al. Choosing clinically interpretable summary measures and robust analytic procedures for quantifying the treatment difference in comparative clinical studies. Stat Med. 2021; 40(28):6235-6242. doi:10.1002/sim.8971.

○ The proposed analysis can easily be conducted using the survRM2 R package: https://cran.r-project.org/web/packages/survRM2/index.html.

- We do agree that restricted mean time-to-event estimates to compare two groups would be advisable in a longitudinal clinical trial when unpredictable intercurrent events (including uncontrollable COVID-19 outbreaks) prevent full observation of study endpoints during follow-up. However, the same model for quantifying the difference and using robust estimation procedures to draw primary inferences seems more appropriate when applied to prospective randomized clinical trials whose design relies on a prespecified statistical hypothesis (e.g., prespecified hazard ratio for comparison between standard and experimental treatment). In this regard, our research, while having a prospective design, is purely an observational evaluation in which we could not envisage a comparison between groups of predefined numerosity nor a comparison on the basis of a predetermined hazard ratio. Although additional statistical evaluations suggested by the Reviewer are convincing and sustainable, for the reasons stated above, we do not believe that they are properly applicable to our clinical study.

• Line 243: Please cite the reference rate of breakthrough infection in the relevant population (not only that the rate was higher in this study).

- As the Reviewer suggested, we reported for proper comparison the rate of breakthrough infections (13.6%) in the reference population (anti-SARS-CoV-2 vaccinated cancer patients on active treatment)

• Line 293: Was there any evidence of heterogeneity in the association between antibody response and odds of breakthrough infection, or between breakthrough infection and time to treatment failure, across the different types of cancer?

- In the initial analysis of the study, we did not consider cancer-specific diagnosis as a potential covariate in the uni- and multivariate testing because of the consistent heterogeneity among the different types. However, reference data suggest that the incidence of breakthrough infections may vary significantly among different tumor types (lower in breast cancer and higher in patients with lung or colorectal cancer) [Wang, W.; Kaelber, D.C.; Xu, R.; Berger, N.A. Breakthrough SARS-CoV-2 infections, hospitalizations, and mortality in vaccinated patients with cancer in the US between December 2020 and November 2021. JAMA Oncol. 2022, 8, 1027–1034]. For this reason and following the Reviewer's suggestions, we included cancer diagnosis [breast (reference category), lung, colorectal, and other cancers] among the covariates in the multivariate analysis of breakthrough infections and treatment failure after booster vaccination. We observed that lung cancer diagnosis was an independent factor in raising the likelihood of COVID-19 outbreaks according to multivariate analysis. The same covariant, however, was not correlated with a significantly reduced time-to-treatment failure after booster vaccination at multivariate testing. Accordingly, we modified every relevant part of the article, including the tables and figures. In addition, we included a supplementary table (Suppl. Table 1) in which we described the univariate comparative assessment of anti-RBD-S1 IgG titers as a function of different covariates (including types of cancer diagnoses) and multivariate analysis by fitting a generalized linear model on the logarithmic values of antibody titers as a function of the same covariates.

Reviewer 3 Report

The article "Effects of antibody response after booster vaccination on SARS-CoV-2 breakthrough infections and disease outcomes in advanced cancer patients: A prospective analysis of the Vax-On-Third study" shows that the Authors enrolled 230 cancer patients who were on active treatment for advanced disease and had received booster dosing of mRNA-BNT162b2 vaccine as part of the Vax-On-Third trial between September 2021 and October 2021. The final data support the role of vaccine boosters in preventing the incidence and severity of COVID- 19 outbreaks. Enhanced humoral immunity after the third vaccination significantly correlates with protection against breakthrough infections.

This study shows very interesting data, with excellent writing quality. The methods and the statistical analysis are clearly detailed. The conclusions are well supported by the results. The quality and clarity of three tables and figure 2 is good. Figure 1 and above all figure 3 are not immediately interpretable, but satisfactory. The references are appropriate and current.

Some minor issues should be addressed before this manuscript can be considered for publication:

1. The introduction is well presented, but it seems too short.

2. The Authors might consider moving Figure 3 to the supplementary material.

Author Response

Answers to Comments and Suggestions for Authors: Reviewer 3

The article "Effects of antibody response after booster vaccination on SARS-CoV-2 breakthrough infections and disease outcomes in advanced cancer patients: A prospective analysis of the Vax-On-Third study" shows that the Authors enrolled 230 cancer patients who were on active treatment for advanced disease and had received booster dosing of mRNA-BNT162b2 vaccine as part of the Vax-On-Third trial between September 2021 and October 2021. The final data support the role of vaccine boosters in preventing the incidence and severity of COVID- 19 outbreaks. Enhanced humoral immunity after the third vaccination significantly correlates with protection against breakthrough infections.

This study shows very interesting data, with excellent writing quality. The methods and the statistical analysis are clearly detailed. The conclusions are well supported by the results. The quality and clarity of three tables and figure 2 is good. Figure 1 and above all figure 3 are not immediately interpretable, but satisfactory. The references are appropriate and current.

Some minor issues should be addressed before this manuscript can be considered for publication:

1. The introduction is well presented, but it seems too short.

- We agree that the introduction, though comprehensive, is actually too brief. Following the Reviewer's suggestions, we have described in more detail the preliminary considerations that led to the different evaluations of the study. In response to another Reviewer's comments, we have provided a specific description of the heterogeneous risk of COVID-19 breakthrough infections in different types of advanced malignancies on active treatment.

2. The Authors might consider moving Figure 3 to the supplementary material.

- As suggested, we have moved Figure 3, previously edited in response to another Reviewer's comments, from the main text to the Supplementary Materials as Supplementary Figure 1.