Remdesivir in the Treatment of COVID-19: A Propensity Score-Matched Analysis from a Public Hospital in New York City Assessing Renal and Hepatic Safety

Round 1

Reviewer 1 Report

Lim et al. used a propensity score analysis to assess the renal and hepatic safety of remdesivir in the treatment of COVID 19. From a population of 927 patients, they matched for a cohort of 248 patients (124 in each group) and found that patients treated with remdesivir had statistically significant lower rates of acute kidney injury. Lower rates of acute liver injury and death was also observed but the difference did not reach statistical significance. Overall, the manuscript was well written and structured. The manual adjudication of 14 covariates for propensity score should be commended.

Major suggestions include:

• From the background, the importance of this study lies in the "real world assessment" of remdesivir safety in terms of renal and hepatic safety. However, similar studies have been conducted- please discuss how this study fits in the context of these similar studies.

Thakare S, Gandhi C, Modi T, Bose S, Deb S, Saxena N, Katyal A, Patil A, Patil S, Pajai A, Bajpai D, Jamale T. Safety of Remdesivir in Patients With Acute Kidney Injury or CKD. Kidney International Reports. 2021;6(1):206-10. doi: 10.1016/j.ekir.2020.10.005.

Wu B, Luo M, Wu F, He Z, Li Y and Xu T (2022) Acute Kidney Injury Associated With Remdesivir: A Comprehensive Pharmacovigilance Analysis of COVID-19 Reports in FAERS. Front. Pharmacol. 13:692828. doi: 10.3389/fphar.2022.692828

Seethapathy R, Zhao S, Long JD, Strohbehn IA, Sise ME. A Propensity Score-Matched Observational Study of Remdesivir in Patients with COVID-19 and Severe Kidney Disease. Kidney360. 2021 Dec 3;3(2):269-278. doi: 10.34067/KID.0006152021. PMID: 35373125; PMCID: PMC8967642.

• please define KDIGO AKI definition used- just creatinine criteria or UOP criteria too, Creatinine increase > 0.3 over 48 hours included in AKI definition?
• As AKI is a primary outcome, recommend evaluating for stage 2 vs 3 AKI.
• Please discuss missing data
• Please explicitly state how COVID-19 severity was staged rather than just citing based on NIH COVID treatment guidelines. It looks like patients were staged by respiratory parameters and signs of (new?) end organ dysfunction. Does that include AKI/ALI? Could that potentially impact the validity of results? manuscript could be improved by highlighting the benefits as well as pitfalls of propensity analysis:

 Leisman, Daniel E. BS1,2 Ten Pearls and Pitfalls of Propensity Scores in Critical Care Research: A Guide for Clinicians and Researchers, Critical Care Medicine: February 2019 - Volume 47 - Issue 2 - p 176-185 doi: 10.1097/CCM.0000000000003567

Minor modifications:

• Typo: blood urea rather than (ureal) nitrogen
• CKD staging: describe which equation used for eGFR calculation
• Would explicitly state in methods that CLD categorized as alcoholic, hepatitis B and C, cirrhosis categorized was none, compensated, decompensated

Author Response

Comment 1: Lim et al. used a propensity score analysis to assess the renal and hepatic safety of remdesivir in the treatment of COVID 19. From a population of 927 patients, they matched for a cohort of 248 patients (124 in each group) and found that patients treated with remdesivir had statistically significant lower rates of acute kidney injury. Lower rates of acute liver injury and death was also observed but the difference did not reach statistical significance. Overall, the manuscript was well written and structured. The manual adjudication of 14 covariates for propensity score should be commended.

Response 1: We would like to thank Reviewer #1 for their very kind comments.

 

Comment 2: From the background, the importance of this study lies in the "real world assessment" of remdesivir safety in terms of renal and hepatic safety. However, similar studies have been conducted- please discuss how this study fits in the context of these similar studies:

Thakare S, Gandhi C, Modi T, Bose S, Deb S, Saxena N, Katyal A, Patil A, Patil S, Pajai A, Bajpai D, Jamale T. Safety of Remdesivir in Patients With Acute Kidney Injury or CKD. Kidney International Reports. 2021;6(1):206-10. doi: 10.1016/j.ekir.2020.10.005.

Wu B, Luo M, Wu F, He Z, Li Y and Xu T (2022) Acute Kidney Injury Associated With Remdesivir: A Comprehensive Pharmacovigilance Analysis of COVID-19 Reports in FAERS. Front. Pharmacol. 13:692828. doi: 10.3389/fphar.2022.692828

Seethapathy R, Zhao S, Long JD, Strohbehn IA, Sise ME. A Propensity Score-Matched Observational Study of Remdesivir in Patients with COVID-19 and Severe Kidney Disease. Kidney360. 2021 Dec 3;3(2):269-278. doi: 10.34067/KID.0006152021. PMID: 35373125; PMCID: PMC8967642.

Response 2: We would like to thank Reviewer #1 for this thoughtful and constructive comment. We would also like to thank our Reviewer for bringing up these studies. We believe that our study, despite its inherent limitations as an observational study, adds to the existing literature for several reasons. For instance, the study from Thakare et al was a significantly smaller study (154 patients), the study from Jamale et al had a different approach being based on data from an adverse event reporting system database, hence was prone to reporting bias, and the study from Seethapathy et al included a very small population (34 patients). We now reference all these studies in the introduction section and we have highlighted with yellow for your review.

 

Comment 3: Please define KDIGO AKI definition used- just creatinine criteria or UOP criteria too, Creatinine increase > 0.3 over 48 hours included in AKI definition?

Response 3: We would like to thank Reviewer #1 for this comment. We used only the criteria (> 0.3 over 48 hours or 1.5 times over 7 days). No accurate urine output data were expected to be available. We specified this information in the methods section and highlighted with yellow for your review.

 

Comment 4: As AKI is a primary outcome, recommend evaluating for stage 2 vs 3 AKI.

Response 4: We would like to thank our Reviewer for this excellent comment. We did a post-hoc analysis looking at the AKI stage before and after matching. The findings are consistent with the original findings and are presented as supplementary table 1 (new). We highlighted the respective changes in methods and results sections in the manuscript with yellow for your review.

 

Comment 5: Please discuss missing data

Response 5: We would like to thank Reviewer #1 for this comment. Thanks to the inpatient acute setting of the study and the already established protocols with regards to specific lab tests follow-up in our institution, we had no missing data. We added a sentence in the section 3.1 and highlighted with yellow for your review. 

 

Comment 6: Please explicitly state how COVID-19 severity was staged rather than just citing based on NIH COVID treatment guidelines. It looks like patients were staged by respiratory parameters and signs of (new?) end organ dysfunction. Does that include AKI/ALI? Could that potentially impact the validity of results?

Response 6: We would like to thank our Reviewer for this constructive comment. Indeed, the NIH criteria used were based on respiratory parameters (SO2) for moderate and severe disease and need for intubation or end-organ dysfunction or ICU admission for critical disease. We specified the criteria in the methods section and highlighted with yellow for your review. Only about 13% of our patients were classified as having critical illness, while the rest (about 87%) as having moderate or severe disease, for whom only respiratory parameters were used based on the above-mentioned NIH classification. Overall, we do not believe that the use of this classification as a variable for our matching has affected our results in significant way.

 

Comment 7: The manuscript could be improved by highlighting the benefits as well as pitfalls of propensity analysis: Leisman, Daniel E. BS1,2 Ten Pearls and Pitfalls of Propensity Scores in Critical Care Research: A Guide for Clinicians and Researchers, Critical Care Medicine: February 2019 - Volume 47 - Issue 2 - p 176-185 doi: 10.1097/CCM.0000000000003567

Response 7: We would like to thank your Reviewer for this comment. We made an addition in the limitation paragraph and also cited this important study.

 

Comment 8: Typo: blood urea rather than (ureal) nitrogen

Response 8: We would like to thank your Reviewer for catching this typographic error, which was corrected and highlighted with yellow for your review.

 

Comment 9: CKD staging: describe which equation used for eGFR calculation

Response 9: We would like to thank our Reviewer for giving us the opportunity to clarify. The CKD-EPI equation was used and is now highlighted in the manuscript (section 2.2) with yellow for your review. 

 

Comment 10: Would explicitly state in methods that CLD categorized as alcoholic, hepatitis B and C, cirrhosis categorized was none, compensated, decompensated

Response 10: We would like to thank Reviewer #1 for this comment. We made the respective modifications in the methods section and highlighted with yellow for your review.

Reviewer 2 Report

JCM 1672339

This is a very well written study of remdesivir in COVID-19 patients. The study had a diverse group of individuals with different comorbidities.

It would be helpful to identify when a participant tested positive of COVID-19 relative to starting remdesivir treatment.

The authors provided good explanations when examining the condition of the patients at time of starting remdesivir that they were more severe cases.

An interesting observation was that Remdesivir  may have provided some renal protection. Can the authors provide any insight based on the available data, when renal function improved with remedesivir?

Do the authors have any data  or insight into inflammatory components and changes with remdesivir?

 

 

Author Response

Comment 1: This was a retrospective study conducted at a public hospital in New York City. The authors desired to assess adverse effects of remdesivir in the treatment of covid-19, specifically the kidney and liver injury risk. Acknowledging that there is a plethora of room for confounding in retrospective studies, a propensity score was created. Patients were then matched based on this score, and logistic regression analyses were run. The results of their models appeared to show a possibility of protection against acute kidney injury in the remdesivir group, although it didn’t meet significance in the matched, multiple regression model. Acute liver injury was also not different between groups; however, this was the case in every model and not just the final matched model. Overall, this was a well-done study in my opinion. The authors set up their question very well in an organized manner. They led with the reasons that they intended to investigate liver and kidney adverse effects, and then they explain why their results were possible in their discussion. The decision to use a propensity score was also commendable. Whether or not to use remdesivir is a decision that different providers answer differently, so controlling for as many factors as possible helps to ease risk of confounding. The results that were found can be logically explained, however there are a few points above that should be addressed so the readers can know exactly who was studied. More information about the population is crucial as Covid-19 treatment has changed so rapidly in the past several years.

Response 1: We would like to thank Reviewer #2 for their kind comments.

 

 

Comment 2: Remdesivir is a generic medication and does not need to be capitalized

Response 2: We would like to thank our Review for this comment. We modified our manuscript accordingly.

 

Comment 3: What was the duration of remdesivir? During the study time period, there was new data emerging about 5 vs 10-day durations

Response 3: We would like to thank our Reviewer for this comment. The institutional guidelines suggested a duration of up to 5 days that could be extended to up to 10 days in patients with critical illness. We added this information in methods section and highlighted for your review.

 

Comment 4: After matching, there was still a significant difference in CKD between the remdesivir yes and no groups, p=0.040. The percentage was lower in the group receiving remdesivir. Could this have impacted the results?

Response 4: We would like to thank our Reviewer for this thoughtful comment. Indeed, the no-remdesivir group had more patients with no CKD compared to the remdesivir group (83.1% vs. 77.4%). Indeed, the matching was not perfect for this variable. It was in favor of the no-remdesivir group with regards to CKD variable. Despite this, we found that the remdesivir group was associated with lower AKI rates. Moreover, the standardized mean difference (SMD) was in favor of balance between groups. Please see the section of statistical analysis in our method section.

 

Comment 5: Was AKI pre-initiation assessed? What time was the baseline SCr taken?

Response 5: We would like to thank the reviewer for this comment. The baseline laboratory tests were obtained while patients were in the emergency room and prior to any treatment initiation as per institutional COVID-19 guidelines. A relevant sentence was added in the methods section and was highlighted with yellow for your review. Regarding the presence of AKI on presentation, this was not possible to be assessed for most of the patients as many patients had been in the hospital for the first time or had not had available kidney function tests for a long period of time (namely their last visit was a significant amount of time before the index visit). 

 

Comment 6: Baseline ALP, AST, and ALT were different after matching between the two groups, could this have had any impact on results?

Response 6: We would like to thank our Reviewer for being so thorough and insightful. Indeed, there are minor and variable differences in the median values of the liver enzymes. These differences are not in favor of the one or the other group consistently. Most importantly, the SMD was suggestive of balance between groups for these variables and perhaps SMD is a better indicator of balance for this situation. Please also see the relevant part in the statistical analysis paragraph of the methods section. Therefore, we do not think that these unavoidable minor differences have meaningfully impacted our results.

 

Comment 7: Is it possible that the no-remdesivir group was sicker? ICU admission higher in the non-remdesivir group after matching there was more intubation, and there was more critically-labeled Covid-19 severity upon admission. However, none of these variables were significant after matching

Response 7: We would like to thank your Reviewer for another thoughtful comment. Indeed, there were more patients with critical disease in the no-remdesivir group, while there were more patients with severe disease in the remdesivir group. Indeed, these differences were not significant after matching indicating a relatively successful matching. Regardless, for robustness purposes, we proceed with logistic regression analyses adjusting for COVID-19 severity on presentation (please see tables 5-7), which largely supported the results of the propensity score matching.

 

Comment 8: Were any patients given other concomitant therapy? Did anyone receive dexamethasone or plasma? Did anyone receive monoclonal antibodies?

Response 8: We would like to thank our Reviewer for being so thorough as this makes our manuscript better. The vast majority of the hospitalized patients received Dexamethasone as per institutional guidelines for COVID-19. Some patients received other treatments, such as Tocilizumab, depending on their course. We have not been able to account for all possible confounders as it was impossible to match for more variables without sacrificing further decrease in our sample size and reduction in the power of our study. We do not have any reason to believe that one group was treated favorably against the other, however we recognize that this is a limitation of our study. Therefore, we added a relevant sentence in the limitations section and highlighted with yellow for your review.

 

Comment 9: What was the vaccination status of the two cohorts?

Comment 9: We would like to thank our Reviewer for this comment. We enrolled patients from June 2020 to early March 2021, when vaccines either were not available or had not been rolled out extensively in the general population. We can confirm that none of our included patients were fully vaccinated.

Reviewer 3 Report

This was a retrospective study conducted at a public hospital in New York City. The authors desired to assess adverse effects of remdesivir in the treatment of covid-19, specifically the kidney and liver injury risk. Acknowledging that there is a plethora of room for confounding in retrospective studies, a propensity score was created. Patients were then matched based on this score, and logistic regression analyses were run. The results of their models appeared to show a possibility of protection against acute kidney injury in the remdesivir group, although it didn’t meet significance in the matched, multiple regression model. Acute liver injury was also not different between groups, however this was the case in every model and not just the final matched model.

 

The questions/comments I have for the authors include:

• Remdesivir is a generic medication and does not need to be capitalized
• What was the duration of remdesivir? During the study time period, there was new data emerging about 5 vs 10 day durations.
• After matching, there was still a significant difference in CKD between the remdesivir yes and no groups, p=0.040. The percentage was lower in the group receiving remdesivir. Could this have impacted the results?
• Was AKI pre-initiation assessed? What time was the baseline SCr taken?
• Baseline ALP, AST, and ALT were different after matching between the two groups, could this have had any impact on results?
• Is it possible that the no-remdesivir group was sicker? ICU admission higher in the non-remdesivir group after matching there was more intubation, and there was more critically-labeled Covid-19 severity upon admission. However none of these variables were significant after matching.
• Were any patients given other concomitant therapy? Did anyone receive dexamethasone or plasma? Did anyone receive monoclonal antibodies?
• What was the vaccination status of the two cohorts?

 

Overall this was a well-done study in my opinion. The authors set up their question very well in an organized manner. They led with the reasons that they intended to investigate liver and kidney adverse effects, and then they explain why their results were possible in their discussion. The decision to use a propensity score was also commendable. Whether or not to use remdesivir is a decision that different providers answer differently, so controlling for as many factors as possible helps to ease risk of confounding. The results that were found can be logically explained, however there are a few points above that should be addressed so the readers can know exactly who was studied. More information about the population is crucial as Covid-19 treatment has changed so rapidly in the past several years.

Author Response

Comment 1: This is a very well written study of remdesivir in COVID-19 patients. The study had a diverse group of individuals with different comorbidities.

Response 1: We would like to thank Reviewer #3 for their very kind comment.

 

 

Comment 2: It would be helpful to identify when a participant tested positive of COVID-19 relative to starting remdesivir treatment.

Response 2: We would like to thank our Reviewer for this comment. All participants had to have a positive COVID-19 PCR to be enrolled. Based on our institutional guidelines for COVID-19, all patients visited the ER were tested for COVID-19 upon arrival to ER and no patients were boarded to medicine (admitted) without known PCR results. Finally, ID stewardship would not give approval for Remdesivir prior to confirmation of COVID-19. Therefore, we are more than confident that all patients were started on Remdesivir after COVID-19 confirmation that was very quick. An addition in the section 2.1 of methods was added and is highlighted with yellow for your review.

 

Comment 3: The authors provided good explanations when examining the condition of the patients at time of starting remdesivir that they were more severe cases.

Response 3: We would like to thank your Reviewer for this kind comment.

 

Comment 4: An interesting observation was that Remdesivir may have provided some renal protection. Can the authors provide any insight based on the available data, when renal function improved with remdesivir?

Response 4: We would like to thank our Reviewer for this great comment. We performed a post-hoc analysis triggered by this comment, which revealed: 1. Mean serum creatinine was decreased from 1.37 before treatment with remdesivir to 1.21 mg/dL after completion of treatment with Remdesivir in the cohort before matching (p<0.001) and from 1.34 mg/dL to 1.19 mg/dL in the cohort after matching (p<0.001). This additional information was included in the section 3.3 and was highlighted with yellow for your review.

 

Comment 5: Do the authors have any data or insight into inflammatory components and changes with remdesivir?

Response 5: We would like to thank our Reviewer for this comment. When we designed the study, we decided to focus on our primary objectives that were to explore the renal and hepatic safety of remdesivir, therefore we did not pursue exploring the impact of remdesivir in inflammatory markers. At some point while designing the study, we thought about using CRP on presentation as a variable indicating severity of disease on presentation but we opted to use the variable “COVID-19 severity on presentation” based on NIH treatment guidelines as being more objective. In brief, unfortunately we do not have this data. 

Round 2

Reviewer 1 Report

Edits reviewed and appropriate. Thank you!