A Detecting System for Abrupt Changes in Temporal Incidence Rate of COVID-19 and Other Pandemics

Round 1

Reviewer 1 Report

The authors aim to create an alert system for pandemics which is a strength.  The model seems to work better for H1N1 in Hong Kong than for Covid -19. The model is relatively simplistic.

 

Major:  Incidence rates, especially at the beginning are a function of testing.  Especially in the case of COVID-19 in the US, testing was extremely limited with only people for who risk of serious morbidity and mortality were tested.  Similarly with the widespread use of at-home lateral flow devices, many positive infections go unreported, thus biasing incidence rates. 

 

Incidence number also needs to take into account the population density.  As a hypothetical example 100 infections in a city the size of Los Angeles is very different story than in a rural town with a population of say 300.  The authors mention this when talking about the results of California, but their model does not take this into account overall.

 

Pandemic velocity within an epidemiologic framework has been explored by others.

Their definition of Rt is very simplistic.  There is often a lag between becoming infectious and the positive test being reported as well the length of the infectious period and as how many susceptible people, on average, a person comes into contact with and then goes on to become infected.  The weights seem to be their attempt to account for this.

 

Seasonality is also not accounted for.

 

The model seems to work well for Hong Kong H1N1 and less well for Covid-19.  It looks like there are a bunch of false positive and false negative signals and it seems like it would be hard to make policy recommendations based on it.  There are a lot of green liens before infections peak again.  For example AZ in 2021 and 2022.  IT looks like the alert is missed just before the huge spike in 2022.   There is also an alert 2 towards the end of the data period that did not lead to a big spike.  

 

Minor:  Page X lines 257, 258.  Los Angeles is misspelled.

 

Why did the authors choose not to use the Johns Hopkins tracker for all of the regions as they track worldwide.

 

How were the 6 states in the US chosen?  Similar question for other countries. Was it an random?  The authors stated that California was not a good fit.  Were these the locations for which the model had a good fit?

Minor editing needed.

Author Response

Thank you so much for your comments! Our response is attached

Author Response File: Author Response.docx

Reviewer 2 Report

The paper presents an interesting detecting system in detecting the abrupt increases and decreases in temporal incidence rate of infection. This system has been applied to the COVID-19 data but we can use it for other pandemics. The detecting sytem in very intuitive and works good for real COVID-19 data.

Author Response

Thank you so much for your comments!

Reviewer 3 Report

I congratualte the authors on having devised a statistical model that predicts incidence based onvarying reproduction numbers which can better inform decision makers and others on impending changes in pandemic incidence rates. You have shown  the model's efficacy out on retrospective data and acknowledge current limitations. If this model is further developed to address these limitations it will be an invaluable tool that should be adopted by country and supra-national health decision making bodies in future.

Author Response

Thank you so much for your comments! We will try to develop the algorithm in the future to address those limitations.

Reviewer 4 Report

The authors presented an algorithm to estimate the time-varying reproduction number and predict the incidence numbers during an epidemic, and to set up an alert system to predict sudden increase or decrease in the incidence number or the reproduction number. The paper clearly demonstrates the results and applications of their method. However, I have some major concerns as outlined below.

  Major comments: 1. My main concern is that this paper does not clearly differentiate its approach from Cori et al. 2013 (Am. J. Epid.) paper. Please make it clear what the novel contribution is compared to the Cori et al. paper. 2. The shiny app web tool mentioned in the discussion section did not work for me, it throws an error message. Please check and make sure the tool works. 3. Since the authors used a 7-day rolling average in the analysis, there is one particular thing that needs to be carefully handled, that is the standard deviation specification. For example, if the raw incidence numbers have a variance of sigma^2, then the 7-day rolling average will have a variance of (sigma^2)/7 if the incidence numbers are assumed independent (which they are obviously not). For correlated incidence numbers (which in reality they are), the expression of this variance will be even more complicated. This requires a careful handling and justification of the model assumptions. Please clearly describe the model assumptions, and explain how you overcome this variance specification issue.   Minor comments: 1. Since this paper is inspired by the response to the Covid-19 pandemic, some of the unique and important properties of the Covid-19 pandemic need to be addressed. Firstly, Covid-19 has a severe under-ascertainment bias in the incidence numbers due to the lack of availability of test kits towards the early stages. Secondly, the test turnaround times were severely lagged, and the lag also changed throughout different stages of the pandemic, which can seriously affect any reported incidence number-based analysis.

Author Response

Thank you so much for your comments!

 

Here are our response to the major comments:

1.Here are the novel distributions:

 1). We constant the model to detect the abrupt changes in advance, which can give alerts and notes.

 2). We use the negative binomial distribution rather than the binomial distribution to describe the distribution of the daily number.

2.Thank you so much for your comments. The shiny app web tool works now!

3.We use the 7-day average incidence number but not the raw incidence number as the input data of our model.  Thus, we only need to consider the variance of the 7-day average incidence number but do not need to worry about the variance of the raw incidence number.

 

Here are our response to the minor:

Thank you for your comments! It is true that there are a severe under-ascertainment bias in the incidence numbers at the begining and lagged test. We will try to solve these problems in our future algorithm and models.

Author Response File: Author Response.docx

Round 2

Reviewer 4 Report

I have a minor comment:

Cori et al. 2013 paper uses Poisson (not Binomial as mentioned in the Authors' response notes) distribution to model the daily incidence numbers. In this manuscript, the proposed method uses negative binomial distribution which the Authors' highlight as a novel contribution. Please provide justification to using negative binomial distribution in the manuscript and highlight its importance compared to the Cori et al. 2013 paper which uses Poisson distribution.

Author Response

Thank you so much for your comments. We will add it to our manuscript.