Phased Implementation of COVID-19 Vaccination: Rapid Assessment of Policy Adoption, Reach and Effectiveness to Protect the Most Vulnerable in the US

Abstract

The US and the rest of the world have suffered from the COVID-19 pandemic for over a year. The high transmissibility and severity of this virus have provoked governments to adopt a variety of mitigation strategies. Some of these previous measures, such as social distancing and mask mandates, were effective in reducing the case growth rate yet became economically and administratively difficult to enforce as the pandemic continued. In late December 2020, COVID-19 vaccines were first approved in the US and states began a phased implementation of COVID-19 vaccination. However, there is limited quantitative evidence regarding the effectiveness of the phased COVID-19 vaccination. This study aims to provide a rapid assessment of the adoption, reach, and effectiveness of the phased implementation of COVID-19 vaccination. We utilize an event-study analysis to evaluate the effect of vaccination on the state-level daily COVID-19 case growth rate. Through this analysis, we assert that vaccination was effective in reducing the spread of COVID-19 shortly after the first shots were given. Specifically, the case growth rate declined by 0.124, 0.347, 0.345, 0.464, 0.490, and 0.756 percentage points corresponding to the 1–5, 6–10, 11–15, 16–20, 21–25, and 26 or more day periods after the initial shots. The findings could be insightful for policymakers as they work to optimize vaccine distribution in later phases, and also for the public as the COVID-19 related health risk is a contentious issue.

1. Introduction

In late December 2019, a pneumonia-like illness was reported in Wuhan, China, which came to be known as COVID-19. In the US, the first case was documented in late January in Washington state. By mid-March, all 50 states had reported cases and by mid-April, the US had the most COVID-19 deaths in the world [1]. To combat the rapidly growing pandemic, many states implemented lockdown measures that coincided with an economic downturn [2]. Therefore, despite growing cases, lockdowns were eased in many states by April [3]. Additionally, there has been a decline in compliance towards social distancing measures which are often voluntary and hard to enforce [4]. As lockdowns are generally not economically sustainable, alternative methods to control the pandemic have received growing attention. Accordingly, many research groups have worked on developing a COVID-19 vaccine. The Centers for Disease Control and Prevention (CDC) has stated that vaccines are vital in tandem with other social distancing measures. In the US, many companies started developing COVID-19 vaccines in March [5]. In December, the Food and Drug Administration (FDA) authorized the emergency use of two mRNA vaccines from Pfizer-BioNTech and Moderna. Despite their notoriously rapid development, clinical trials have found both vaccines to be more than 90% effective with only a few, minor side effects [6].

Once the vaccine efficacy was thoroughly tested, the US government started planning distribution efforts through a program known as Operation Warp Speed [5]. The first round of vaccines was delivered to states, territories, and some federal agencies in late December [7,8]. Ultimately, the states are in charge of the local administration of vaccines. However, as the supply is currently limited, the Advisory Committee on Immunization Practices (ACIP) has suggested taking a phased approach to vaccination [9]. This approach was made in an attempt to decrease deaths and serious illnesses while preserving a functioning society and reducing the burden on those who have been disproportionately harmed by the virus [10,11,12]. The federal government anticipates vaccinating 100 million people in the US by the end of February 2021. However, the vaccine rollout has been slow [7], likely due to the limited supply and inefficient distribution [11,13,14]. It is important to rapidly assess the phased vaccination approach in terms of adoption, reach, and effectiveness, despite this low coverage, in order to better plan for vaccination later on. Indeed, an evaluation of the pandemic in California near the end of January 2021 noted that cases [15] and hospitalizations were starting to decline at the same time vaccination coverage was increasing [16].

Similar to social distancing policies, the COVID-19 vaccination has become a contentious issue especially as there is limited direct evidence regarding the effectiveness of vaccination. Many people who protested early pandemic measures, such as stay-at-home orders and mask mandates, are likewise protesting mandatory vaccines [6]. This opposition may be due to firm beliefs in medical autonomy, vaccine conspiracy theories [17], concerns about vaccine side effects, mistrust of the government [18], and mistrust of healthcare [19]. However, COVID-19 vaccine opinions and knowledge have varied over ime. There has been a noticeable increase in the public’s willingness to take the vaccine as it has moved into later stages of development [6,18]. Increasing the public trust in COVID-19 vaccines is critical as many experts predict that a vaccination rate between 70% and 80% is needed for herd immunity [20]. As manufacturing and distribution become more systematic, it will become more important to eliminate other barriers to widespread vaccination efforts such as public opposition.

In order to increase the public’s willingness to take the vaccine, some studies have recommended utilizing public information campaigns, financial investments, and other measures to highlight the effectiveness, benefits, and safety of the vaccine [21]. Furthermore, other studies have emphasized the importance of making sure these campaigns receive high coverage and are spread to the public quickly, as the pandemic is an urgent issue [22]. Without sufficient knowledge, it is often hard to persuade people to accept recommendations [23]. For example, mask mandates ultimately gained more widespread acceptance once thorough research highlighted their success [24]. Indeed, as the vaccine supply increases, distribution will move towards phases that include a broader range of people who are eligible for vaccination [11]. Therefore, evaluating the effectiveness of vaccination in controlling the spread of COVID-19 is important to ensure a smooth transition between phases by increasing vaccination acceptance and coverage.

Currently, there is limited research done to quantitatively evaluate the effectiveness of the phased COVID-19 vaccination approach. Through a natural experiment, we investigate how statewide vaccination efforts affect the spread of COVID-19 in the US during the early stages of vaccination. Specifically, we use an event-study analysis to estimate the effects of state vaccinations on the state-level confirmed case growth rate over different time periods. Vaccination may play a vital role as states are reopening and social distancing policies are becoming less stringent [4,25]. It is necessary to do a rapid assessment of vaccination effectiveness during the initial stages of implementation as concrete evidence of success and safety is important to inform policymakers who direct administration efforts, and the public in order to achieve widespread vaccination.

2. Methods

2.1. Study Area and Data

2.1.1. Vaccination Data

Records detailing when the COVID-19 vaccine was first administered in every state were gathered in this study by searching and reviewing the public news and governmental pages. Our study focused on state-level COVID-19 vaccination beginning with the first shot which was administered in New York on 14 December 2020. Following the CDC guidelines, frontline healthcare workers and long-term care facility staff and residents were given the highest priority in receiving the vaccine in phase one [12]. However, the vaccination process, including the definition of groups given priority, has varied among the states. Detailed information on the adoption of vaccination in different US states can be found in the result section.

In addition, we collected vaccination data from 1 October, 2020 to 26 January, 2021. A repository shared by the Johns Hopkins University Centers for Civic Impact COVID-19 data analysis and visualizations (https://github.com/govex/COVID-19 (accessed on 30 January 2021)) contains state-level reports which record the daily cumulative number of doses administered to the public. Specifically, we divided the cumulative number of doses administered by a state’s population size to find the daily ratio of doses administered.

2.1.2. State-Level Policy Data

Statewide policy data was collected from the Oxford COVID-19 Government Response Tracker project [26]. This dataset describes common measures governments have taken in the fight against COVID-19 and their implementation over time. These policies include school closures, workplace closures, the cancellation of public events, restrictions on gatherings, public transport closures, stay-at-home orders, internal movement restrictions, and international travel controls. The extent of government action is reflected using a stringency index ranging from 0 to 100; the larger the value, the more stringent a certain policy measure is. The dataset can be found on Github (https://github.com/OxCGRT/covid-policy-tracker (accessed on 30 January 2021)). Numerous studies have found that social distancing policies and mask mandates affect the growth rate of COVID-19 cases [24,27,28], thus, the stringency index was included in this study to control for the effects of policy stringency during the study period.

2.1.3. State-Level Confirmed Case Data

The NSF spatiotemporal center provided a data repository detailing daily COVID-19 data [29,30]. Specifically, our study used the US state-level confirmed case data. This level of data was used as most vaccination plans were designed and implemented by individual states [7,8]. Only confirmed case data was used in this research since we focused on the impact of vaccines on the spread of COVID-19. The daily confirmed case data was gathered from this Github repository (https://github.com/stccenter/COVID-19-Data/tree/master/US (accessed on 30 January 2021)), and daily growth rates of confirmed cases were derived from the original dataset as introduced in the methodology section.

2.2. Data Processing

2.2.1. Vaccination

State-level vaccination data was encoded as binary data, with 0 indicating that the COVID-19 vaccine had not been administered in a certain state and 1 indicating that people had started to take the vaccine in that state. To assess the vaccination effects, a reference group representing the 1–5 days before the start of vaccination was selected. Additionally, we analyzed the effects of the vaccination over six post-periods (1–5 days, 6–10 days, 11–15 days, 16–20 days, 21–25 days, and 26+ days)— representing the days after the first shot of the COVID-19 vaccine was administered—and four pre-event periods (6–10 days, 11–15 days, 16–20 days, and 20+ days)—representing the days before the vaccination started. Multiple 0–1 indicators were generated to indicate these pre- and post- periods. Each day in the analysis period was assigned with one indicator by comparing t to the time range each indicator represented. Here, t is the cumulative number of days since the first shot of the vaccine was administered in a certain state.

2.2.2. Confirmed Case Growth Rate

To estimate the effect of vaccination, we calculated the daily, state-level confirmed case growth rate, which is the difference between the natural logarithms of the cumulative counts of confirmed cases on a given day and that of the prior day. This difference was then multiplied by 100 to enable the daily growth rate to be given in percentage points so that the estimated regression coefficients could be interpreted as percentage point changes in the growth rate.

$$ConfirmedG{R}_{sd}=\ln \left(confir{m}_{sd}\right)-\ln \left(confir{m}_{s,d-1}\right)$$

(1)

2.3. Statistical Analysis

In this study, we employed a fixed-effects panel regression model incorporating the event-study design to examine the effects of statewide phased COVID-19 vaccination on the spread of COVID-19 in the US. The differences between the availability of vaccines before and after the initial vaccination in the states enabled estimation in the context of a natural experiment. We examined if the trend in the COVID-19 confirmed case growth rate was differentiated before and after the start of vaccination. All the statistical analyses were performed using the statistical software system STATA 16. The response variable in the model was the confirmed case growth rate as defined in Equation (1). The effect of the periods of interest were defined in the vaccination section, with the six post periods after the first shot of the COVID-19 vaccine coded $i=1,\dots ,6$, the four pre-event periods coded $i=-4,\dots ,-1$, and the reference period $i=0$. In addition to the differences in vaccination efforts, we also included the test growth rate, the policy stringency index [26], and the dose administrative ratio as control variables.

In the above regression specification,$vAdoptio{n}_{sdi}$ denotes the indicator variable for the policy of vaccination belonging to the period $i$ for state $s$ and day $d$ (normalized to 0), $vReac{h}_{sd}$ denotes the dose administrative ratio for state $s$ and day $d$. Additionally, $stat{e}_s$ denotes the state fixed effects; $ConfirmedG{R}_{sd}$, $TestG{R}_{sd}$, and $stringenc{y}_{sd}$ denote, respectively, the confirmed case growth rate, the test case growth rate, and the stringency index, for state $s$ and day $d$.

Meanwhile, since the first shot of the vaccine was administered on December 14 and most states started vaccination in the following few days, the effects of vaccination may have been affected by the coinciding holiday season. In the regression specification, we added holiday fixed effects to control for possible confounding brought by the Christmas holiday. We denote the holiday season indicator by $da{y}_d$, where $d=82$ and $d=93$ correspond to 7 days before and after Christmas. To capture the pre-vaccination trends, a linear calendar day trend from 31 October to 13 December was estimated using a fixed-effects panel regression model for the confirmed case growth rate. Then, daily confirmed case growth rates were predicted for each state and for the whole time period based on the estimated pre-trend. The predicted case growth rates for state $s$ and day $d$, denoted by $predG{R}_{sd}$, were included as a covariate in the regression model to control for the effects of the pre-vaccination period. Respecting the clustered nature of the repeated measurements in the panel dataset, we calculated robust standard errors clustered by states.

$$\begin{array}{ll}ConfirmedG{R}_{sd}& =\alpha +{\displaystyle \sum _{i\ne 0}}{\beta }_{i}vAdoptio{n}_{sdi}+{\gamma }_{1}vReac{h}_{sd}+{\gamma }_{2}TestG{R}_{sd}\\ & +{\gamma }_{3}stringenc{y}_{sd}+{\gamma }_{4}predG{R}_{sd}+{\displaystyle \sum _{d=82}^{93}}{\delta }_{d}da{y}_d+stat{e}_s\\ & +{ϵ}_{sd}\end{array}$$

(2)

3. Results

3.1. Vaccination Policy Adoption

Vaccination efforts were guided at the state level. All states started the vaccination of priority groups by 16 December; however, some states initiated their vaccine administration quicker than others.

Table 1. The administration date of the first COVID-19 vaccine shot in every state.

Date	States That Administered Their First Shot
14 December 2020	Arkansas, California, Colorado, Connecticut, Florida, Idaho, Indiana, Iowa, Kansas, Kentucky, Louisiana, Maryland, Michigan, Mississippi, Nebraska, Nevada, New York, North Carolina, North Dakota, Ohio, Pennsylvania, Rhode Island, South Dakota, Texas, West Virginia
15 December 2020	Alaska, Alabama, Arizona, Delaware, Georgia, Hawaii, Illinois, Maine, Massachusetts, Minnesota, Missouri, Montana, New Hampshire, New Jersey, New Mexico, Oklahoma, South Carolina, Utah, Virginia, Vermont, Washington, Wisconsin, Wyoming
16 December 2020	Oregon, Tennessee

shows the specific dates when each state commenced vaccinations. The collected information shows that about half of all states administered their first shot of the COVID-19 vaccine on 14 December 2020, with the overall first shot in the US being given in New York on the same day. Most other states started to vaccinate priority groups in the following days. Detailed information on the dates and news sources can be found in Appendix A (

Table A1. Source news for the first dose of COVID-19 vaccine administered on every state in the US.

State	Data of First Shot	Source
Alabama	15 December 2020	https://www.alabamapublichealth.gov/news/2020/12/15.html
Alaska	15 December 2020	https://www.alaskapublic.org/2020/12/15/covid-19-vaccinations-begin-in-alaska/
Arizona	15 December 2020	https://www.kold.com/2020/12/15/ww-veteran-receives-first-covid-vaccine-arizona/
Arkansas	14 December 2020	https://apnews.com/article/arkansas-health-coronavirus-pandemic-coronavirus-vaccine-asa-hutchinson-720507102ad8af4226aa0d0725c5f28f
California	14 December 2020	https://www.kron4.com/health/coronavirus/first-coronavirus-vaccines-administered-in-ca/
Colorado	14 December 2020	https://www.denverpost.com/2020/12/14/colorado-first-covid-vaccines-fort-collins/
Connecticut	14 December 2020	https://www.ctpost.com/news/article/First-shipment-of-COVID-vaccines-to-arrive-in-CT-15799733.php
Delaware	15 December 2020	https://delawarestatenews.net/coronavirus/first-covid-19-vaccine-administered-in-delaware/
Florida	14 December 2020	https://www.miamiherald.com/news/coronavirus/article247835830.html
Georgia	15 December 2020	https://patch.com/georgia/dallas-hiram/first-doses-coronavirus-vaccine-administered-savannah
Hawaii	15 December 2020	https://www.khon2.com/local-news/hawaiis-first-doses-of-covid-19-vaccine-administered-to-five-frontline-healthcare-workers/
Idaho	14 December 2020	https://apnews.com/article/brad-little-health-coronavirus-pandemic-rexburg-idaho-cfd2a0fc35353dff216fd487705ce55f
Illinois	15 December 2020	https://www.nbcchicago.com/news/coronavirus/illinois-coronavirus-updates-first-doses-of-vaccine-to-be-administered-across-the-state/2394867/
Indiana	14 December 2020	https://www.wishtv.com/news/coronavirus/first-doses-of-covid-19-vaccine-in-indiana-administered-in-fort-wayne/
Iowa	14 December 2020	https://www.thegazette.com/subject/news/health/iowa-covid-vaccine-first-dose-university-of-iowa-hospital-20201214
Kansas	14 December 2020	https://apnews.com/article/topeka-kansas-wichita-coronavirus-pandemic-b082566461165e59965aba9ad20c0de3
Kentucky	14 December 2020	https://www.kentucky.com/news/coronavirus/article247830445.html
Louisiana	14 December 2020	https://apnews.com/article/new-orleans-john-bel-edwards-coronavirus-pandemic-louisiana-94950e5ec2b050460af561afa657ead3
Maine	15 December 2020	https://wgme.com/news/local/the-first-covid-19-vaccines-have-been-administered-in-maine
Maryland	14 December 2020	https://www.baltimoresun.com/coronavirus/bs-hs-umms-gets-covid-vaccine-20201214-n4i4vsebtfcufphbzzchsv45uq-story.html
Massachusetts	15 December 2020	https://www.mass.gov/info-details/massachusetts-covid-19-vaccination-data-and-updates
Michigan	14 December 2020	https://www.detroitnews.com/story/news/local/michigan/2020/12/14/first-covid-19-vaccinations-administered-michigan/6544944002/
Minnesota	15 December 2020	https://www.kare11.com/article/news/health/coronavirus/minneapolis-va-administers-first-covid-vaccine-in-minnesota/89-3f896b65-db8d-4b93-a41b-51851b722208
Mississippi	14 December 2020	https://www.clarionledger.com/story/news/local/2020/12/14/covid-19-vaccine-update-mississippi-msdh-distribution/6539816002/
Missouri	15 December 2020	https://www.stltoday.com/news/local/metro/first-st-louis-area-covid-19-vaccine-doses-administered-hope-kindled-for-region/article_dccee832-f19d-5414-b966-174e451b994e.html
Montana	15 December 2020	https://www.tester.senate.gov/?p=press_release&id=7881
Nebraska	14 December 2020	https://www.wowt.com/2020/12/14/doses-of-pfizer-covid-19-vaccine-arrives-in-omaha-chi-health-works-on-vaccination-plan/
Nevada	14 December 2020	https://www.8newsnow.com/news/health/coronavirus-health/umc-becomes-first-nevada-hospital-to-administer-covid-19-vaccine-hundreds-of-front-line-workers-vaccinated-monday/
New Hampshire	15 December 2020	https://www.governor.nh.gov/news-and-media/photo-release-first-covid-19-vaccine-administered-new-hampshire
New Jersey	15 December 2020	https://www.nj.com/coronavirus/2020/12/nj-administers-its-first-covid-19-vaccine-dose-to-university-hospital-nurse.html
New Mexico	15 December 2020	https://kfoxtv.com/news/coronavirus/new-mexico-provides-details-on-covid-19-vaccine-distribution-plan
New York	14 December 2020	https://abc7ny.com/covid-vaccine-coronavirus-pfizer-ny/8763858/
North Carolina	14 December 2020	https://www.citizen-times.com/story/news/local/2020/12/16/covid-vaccine-north-carolina-distribution-plan-mask-coronavirus/3921469001/
North Dakota	14 December 2020	https://www.kxnet.com/news/health/coronavirus/dr-avish-nagpal-is-the-first-person-in-north-dakota-to-receive-coronavirus-vaccine/
Ohio	14 December 2020	https://wexnermedical.osu.edu/mediaroom/pressreleaselisting/ohio-state-among-first-to-administer-covid-vaccine#:~:text=For%20the%20Media-,Ohio%20State%20Among%20First%20in%20U.S.%20and%20First,to%20Administer%20COVID%2D19%20Vaccine&text=COLUMBUS%2C%20Ohio%20%E2%80%93%20The%20Ohio%20State,administer%20the%20COVID%2D19%20vaccine
Oklahoma	15 December 2020	https://www.fox23.com/news/local/covid-19-vaccine-arrives-oklahoma/U6SVLPKSZBBILJ5D6QBUFIKGUI/
Oregon	16 December 2020	https://apnews.com/article/science-ontario-coronavirus-pandemic-oregon-181bc863029023d64c72a45fc8c2ef7f
Pennsylvania	14 December 2020	https://www.pennlive.com/news/2020/12/first-covid-19-vaccines-administered-in-pennsylvania.html
Rhode Island	14 December 2020	https://www.wpri.com/health/coronavirus/ri-health-care-workers-first-to-receive-covid-19-vaccine/
South Carolina	15 December 2020	https://www.postandcourier.com/health/covid19/first-south-carolina-coronavirus-vaccines-administered-to-health-care-workers/article_efc5e38a-3bf9-11eb-a85d-9fbfd1f767eb.html
South Dakota	14 December 2020	https://www.dakotanewsnow.com/2020/12/15/covid-19-vaccine-arrives-in-south-dakota-first-immunizations-administered/
Tennessee	16 December 2020	https://www.tennessean.com/story/news/health/2020/12/16/covid-19-vaccinations-begin-but-many-tennessee-hospitals-must-wait/3909694001/
Texas	14 December 2020	https://www.houstonpublicmedia.org/articles/news/health-science/coronavirus/2020/12/14/387843/first-doses-of-covid-19-vaccine-begin-to-arrive-in-texas/
Utah	15 December 2020	https://kutv.com/news/local/first-covid-19-vaccine-administered-in-utah-to-frontline-workers
Virginia	15 December 2020	https://www.wavy.com/news/health/coronavirus/norfolk-general-to-give-out-first-covid-19-vaccine-doses-tuesday-northam-to-observe/
Vermont	15 December 2020	https://www.necn.com/news/local/vt-gov-to-give-coronavirus-update-12/2370806/
Washington	15 December 2020	https://www.kxly.com/first-covid-19-vaccinations-administered-to-washington-health-care-workers/
West Virginia	14 December 2020	https://www.wvnews.com/news/wvnews/first-doses-of-covid-19-vaccine-administered-in-west-virginia/article_aa15cdaf-a477-56ed-bb99-036dc132d04a.html
Wisconsin	15 December 2020	https://www.wisn.com/article/covid-19-wisconsin-nurses-among-first-to-get-vaccine/34949672#

).

The delivery of vaccinations to the public was implemented in settings such as healthcare facilities, stadiums, and pharmacies, which received doses from the state [31]. Since the supply of vaccines is limited, the ACIP suggests offering vaccination in a phased approach [10]. This approach gives vulnerable populations a high priority. According to the recommendation, Phase 1a includes healthcare personnel and long-term care facility residents. Phase 1b includes persons ≥75 years of age and frontline essential workers. Phase 1c includes persons 65–74 years of age, persons 16–64 years of age with high-risk medical conditions, and other essential workers. However, as distribution was left up to individual states, many phases were defined slightly differently. The CDC phase definition that most closely aligned to the state’s most recently vaccinated populations (as of 26 January 2021) was used to categorize the state in

Table 2. The implementation status of COVID-19 vaccination in every state (as of 26 January 2021).

Phase	States in a Certain Phase
Phase 1a	Arizona, Florida, Georgia, Hawaii, Kentucky, Maine, Oregon, Utah
Phase 1b	Arkansas, California, Colorado, Delaware, Idaho, Illinois, Indiana, Iowa, Kansas, Louisiana, Massachusetts, Michigan, Montana, New Mexico, North Carolina, North Dakota, Ohio, Oklahoma, Rhode Island, South Carolina, Tennessee, Vermont, Washington, Wyoming
Phase 1c	Alabama, Alaska, Connecticut, Maryland, Minnesota, Mississippi, Missouri, Nebraska, Nevada, New Hampshire, New Jersey, New York, Pennsylvania, South Dakota, Texas, Virginia, West Virginia, Wisconsin

, which represents the state-level implementation of vaccination. Information regarding each state’s specific vaccination phases was gathered from USAFacts (https://usafacts.org/visualizations/covid-vaccine-tracker-states (accessed on 30 January 2021)). The states also show variation in their vaccination progress. Although all states are still in phase 1 as of 26 January, some states have already entered phase 1c while others are still in phase 1a.

3.2. Vaccination Reach

Figure 1 presents the cumulative number of administered doses in each state by certain dates during the analysis period. Throughout this time, some states saw rapid growth in the number of doses they administered. For example, Texas and Florida slowly started vaccination on 14 December 2020, as was the case with most other states during the start of vaccination efforts. However, by 26 January 2021, Florida and Texas had some of the highest totals of administered doses. Many states did not see such a significant change throughout the analysis period. Arkansas, for example, had low numbers of administered doses during the whole period. Over time, Texas and Florida likely became more efficient in their vaccination efforts.

The attempts of each state to reach the public can also be seen in Figure 2, which shows the percentage of the population, in each US state who received at least 1 vaccine dose by the same four dates described above. Throughout the analysis period, Texas and Florida were able to notably increase the percentage of their populations who received at least 1 dose of the vaccine. West Virginia also saw large increases in this population percentage and has been commended for its vaccination efforts [32]. In terms of vaccinations, some states have been more effective in reaching the public as seen by changes in the number of doses administered and the percent of the population with at least 1 vaccine dose over time.

3.3. Vaccination Effectiveness

Figure 3 displays the effects of statewide COVID-19 vaccination on the state-level daily confirmed case growth rates based on the fixed effects panel regression with an event-study design specified in Equation (2). In addition, changes in the daily growth rate percentage for the six post-periods and the four pre-event periods with respect to the reference period can also be visualized in Figure 3. The red vertical line in Figure 3 represents the reference period and the black square symbols describe the point estimates (along with their 95% confidence intervals) of the vaccination effect over different time periods. The figure shows that there were significant decreases in the daily COVID-19 case growth rate after the initial shots of the vaccine, with 0.124, 0.347, 0.345, 0.464, 0.490, and 0.756 percentage point declines in the 1–5, 6–10, 11–15, 16–20, 21–25, and 26 or more day post-periods after the start of vaccination.

3.4. Robustness Check

To verify the robustness of the estimates from the regression model, robustness checks were performed using different controls and samples (

Table 3. The impact of vaccination on COVID-19 confirmed cases: Results from different models.

	(1) Main Results (Controlling for Christmas Holidays and Pre-Vaccination Trend)	(2) Controlling for Christmas Holidays	(3) Controlling for Pre-Vaccination Trend	(4) Without Controlling Holiday and Pre-vaccination Trend	(5) Excluding AL	(6) Excluding WV	(7) Excluding NM, WI, DE, TN, AK
21 or More Days Before	−0.025	0.803 ***	−0.025	0.802 ***	−0.022	−0.024	−0.026
16 to 20 Days Before	0.001	0.303 ***	0.001	0.303 ***	0.003	0.001	0.015
11 to 15 Days Before	0.033	0.333 ***	0.034	0.333 ***	0.034	0.033	0.035
6 to 10 Days Before	0.042	0.269 ***	0.042	0.269 ***	0.043	0.043	0.036
1 to 5 Days After	−0.124 ***	−0.291 ***	−0.124 ***	−0.291 ***	−0.137 ***	−0.133 ***	−0.107 ***
6 to 10 Days After	−0.347 ***	−0.409 ***	−0.590 ***	−0.650 ***	−0.354 ***	−0.392 ***	−0.310 ***
11 to 15 Days After	−0.354 ***	−0.445 ***	−0.649 ***	−0.736 ***	−0.360 ***	−0.412 ***	−0.275 **
16 to 20 Days After	−0.464 ***	−0.585 ***	−0.482 ***	−0.599 ***	−0.473 ***	−0.488 ***	−0.431 ***
21 to 25 Days After	−0.490 ***	−0.625 ***	−0.488 ***	−0.621 ***	−0.501 ***	−0.514 ***	−0.460 ***
26 or More Days After	−0.756 ***	−0.951 ***	−0.752 ***	−0.943 ***	−0.771 ***	−0.755 ***	−0.727 ***
Stringency	−0.002	0.014 **	−0.002	0.014 **	−0.002	−0.002	−0.001
TestGR	0.199	−0.078	0.2	−0.077	0.199	0.198	0.191
vReach	−0.069	−0.05	−0.07	−0.052	−0.066	−0.067	−0.077
_cons	0.464	0.532	0.464	0.53	0.436	0.464	0.438

Notes: ** p < 0.05, *** p < 0.01; _cons, constant term; AL, Alabama; WV, West Virginia; NM, New Mexico; WI, Wisconsin; DE, Delaware; TN, Tennessee; AK, Alaska.

). The main results column in Table 1 shows the effects of vaccination on the case growth rate with controlling for the Christmas holidays and pre-vaccination trends as explained previously. Again, this is the main regression we adopted, of which the average vaccine effects were plotted in Figure 3. Three other versions of the model were run: controlling for the Christmas holidays, controlling for the pre-vaccination trend, and without controlling for the Christmas holidays, as shown in Columns 2, 3, and 4 respectively. These results showed similar effects on the case growth rate as seen with the main result. Columns 5–7 are robustness checks using the main regression specification but with some states excluded. Alabama was excluded in Column 5 because it has one of the lowest vaccination rates, while West Virginia was excluded in Column 6 because it has one of the highest vaccination rates. New Mexico, Wisconsin, Delaware, Tennessee, and Alaska were excluded from the test in Column 7 as these states represented a wide range in their percentage of distributed vaccines that have been administered [33]. The results are robust in regard to these exclusions as they display similar results to what was seen in the main analysis.

4. Discussion

Vaccination efforts in the US commenced in mid to late December. Through the evaluation of certain periods past this initial adoption, our study showed that vaccination could significantly decrease the case growth rate in short-term, subsequent time periods (Figure 3). Furthermore, as vaccines become more widely available globally, this model and the insights gleaned from this study, could be extrapolated to other countries where vaccination is initiated to protect the most vulnerable populations, such as healthcare personnel, long-term care facility residents and the elderly in general, frontline essential workers, and adults with high-risk medical conditions. This study is especially relevant as economies have started to reopen and social distancing compliance is declining, while cases are still at some of their highest totals.

Since the early stages of the pandemic, governments have distributed countless resources to support the development of the COVID-19 vaccine. Indeed, the vaccine has been claimed to be a vital tool in fighting the virus among other mitigation strategies and social distancing measures. However, the early administration of vaccines has been met with notable contention in the US. A similar mistrust of vaccination can be found in countries beyond the US, such as in the UK [34]. The hesitancy to adopt vaccination in these countries has been partly attributed to the spread of misinformation, especially scientific-sounding misinformation [34]. Much public uncertainty can be attributed to a lack of quantitative evidence regarding how vaccination affects COVID-19 spread, case severity, and case mortality at the population level. As our study highlights the effectiveness of phased vaccination in the US, this result is important in gaining public acceptance and ultimately implementing vaccination at scale.

Although vaccinations are generally effective in reducing the COVID-19 case growth rate, vaccination implementation (Figure 1 and Figure 2) and its effects on the case growth rate varied by state. The magnitude of the decline in the case growth rate may be affected by factors that are heterogeneous among states, such as population density and other variables that have been associated with COVID-19. Additionally, inconsistency in implementation strategies can be seen in how well each state reached the public. For example, West Virginia increased vaccine administration dramatically throughout this analysis period (Figure 2). Successful vaccination in West Virginia was partly attributed to the use of a network of pharmacies that facilitated local distribution [32]. Indeed, vaccination allocation may play a role in reaching different communities [35]. Additionally, as vaccination is a contentious issue and many people are skeptical of the vaccine due to a lack of quantitative information highlighting its effectiveness, many experts have recommended utilizing public information campaigns and media outlets to help spread information. These sources are vital in highlighting the safety and effectiveness of the vaccine to the public [35,36]. Making sure mitigation measures reach the public is an essential step in the ultimate goal of widespread vaccination and the fight against COVID-19. Other states can learn from these successful implementation strategies in order to reach a larger percentage of the public in later phases.

Our study contributes direct, empirical evidence, which highlights the effectiveness of the COVID-19 vaccination on the case growth rate. Overall, vaccination has been successful in decreasing the case growth rate during the early stages of vaccination despite the initial limited supply and slow rollout. As seen in Figure 3, there were no significant changes in the daily case growth rate until after vaccine implementation. Furthermore, the decreases increased in magnitude over time which is promising for the effectiveness of vaccine implementation. It is also important to note that these results were robust although they represented a very early stage of vaccine administration. Controlling for holidays, pre-vaccination trends, and testing on modified sample sets further supported our conclusion that the vaccine has been successful in decreasing the case growth rate shortly after implementation (Table 3). This information could be used to convince the public that vaccination is worthwhile and can help policymakers optimize implementation strategies.

Despite these promising results, this study was limited by the available data. For example, the vaccine technology employed in phase one vaccination implementation is mRNA and we have limited understanding of the population impact of viral vector-based vaccines. The real number of COVID-19 cases is likely much higher than the totals being reported [37]. Future work may also investigate the possible consequences of the vaccine using severe case and death case data. Similarly, the model used in this analysis failed to control for factors such as mask use and seasonality due to data availability. The vaccination approach and system will shift when more COVID-19 vaccines, such as viral vector-based vaccines, are approved by the FDA for regular distribution and supplies become more readily available [38]. Understanding phased vaccination to reach herd immunity and the impact of vaccination on the growth rate of severe cases and mortality at the population level are important. Future research is warranted to estimate the reach, effectiveness, adoption, implementation, and maintenance of regular COVID-19 vaccination.

5. Conclusions

This study found that the phased vaccination approach helped reduce the daily, state-level COVID-19 case growth rates in the US. Despite initially limited vaccine availability, the CDC guidelines regarding the phased vaccination approach have helped protect the most vulnerable from COVID-19. Even though the COVID-19 vaccine is still in the early stages of distribution, it has already been associated with significant decreases in the case growth rate. This is promising for the subsequent phases of vaccination. Overall, the results presented in this research have implications in regard to general public vaccine concerns. For example, our assertion that the COVID-19 vaccine is effective, even in the initial stage, may help persuade skeptical vaccine critics that this mitigation measure is worth it. This information is vital as states are reopening and social distancing measures are becoming less feasible both administratively and economically.