Optimising Clinical Epidemiology in Disease Outbreaks: Analysis of ISARIC-WHO COVID-19 Case Report Form Utilisation

Abstract

Standardised forms for capturing clinical data promote consistency in data collection and analysis across research sites, enabling faster, higher-quality evidence generation. ISARIC and the World Health Organization have developed case report forms (CRFs) for the clinical characterisation of several infectious disease outbreaks. To improve the design and quality of future forms, we analysed the inclusion and completion rates of the 243 fields on the ISARIC-WHO COVID-19 CRF. Data from 42 diverse collaborations, covering 1886 hospitals and 950,064 patients, were analysed. A mean of 129.6 fields (53%) were included in the adapted CRFs implemented across the sites. Consistent patterns of field inclusion and completion aligned with globally recognised research priorities in outbreaks of novel infectious diseases. Outcome status was the most highly included (95.2%) and completed (89.8%) field, followed by admission demographics (79.1% and 91.6%), comorbidities (77.9% and 79.0%), signs and symptoms (68.9% and 78.4%), and vitals (70.3% and 69.1%). Mean field completion was higher in severe patients (70.2%) than in all patients (61.6%). The results reveal how clinical characterisation CRFs can be streamlined to reduce data collection time, including the modularisation of CRFs, to offer a choice of data volume collection and the separation of critical care interventions. This data-driven approach to designing CRFs enhances the efficiency of data collection to inform patient care and public health response.

1. Introduction

Clinical data are the foundation of an evidence-based response to outbreaks—at both the clinical and policy levels. The case definition, time to symptom progression, rates and covariates of clinical outcomes, and risk factors for severe disease or fatality must be rapidly defined to manage novel infections or unexpected changes in the epidemiology of known infections [1]. However, the collection of clinical data relies on the availability and time of front-line health care workers and research staff. This is especially true in low-resource settings where electronic health records are less common and disease outbreaks are more common [2,3]. As health emergencies often cause an increase in clinical caseload and other disruptions to health systems [4,5], it is important to minimise the burden of data collection to avoid detracting from patient care responsibilities. Evidence and innovations are required to design more efficient data tools that continue to address key knowledge gaps while optimising the use of limited staff resources.

For more than a decade, ISARIC (the International Severe Acute Respiratory and emerging Infections Consortium) and the World Health Organization (WHO) have developed case report forms (CRFs) that collect data to characterise the key clinical features of emerging infectious diseases [6]. These forms have been used in response to outbreaks of MERS-CoV, Ebola virus disease, Zika, severe acute respiratory illness, acute non-A-E hepatitis, and Mpox [7,8,9,10,11,12]. In January 2020, upon recognition of the need for urgent research on the novel SARS-CoV-2, ISARIC and WHO adapted their existing data forms and created a paper and electronic CRF to characterise COVID-19 [13]. The form was used by thousands of sites around the world to collect data that informed local, regional, and international pandemic responses [14]. While some sites used the form in its original design [15,16,17,18], others used it as a template to create a locally tailored version [19,20,21]. Requests for support to amend the forms for specific contexts, including low-resource settings and critical care units, resulted in the creation of an ISARIC-WHO ‘Rapid’ COVID-19 CRF and add-on modules with bespoke fields (or questions) for special populations, e.g., pregnant women. All forms and modules were openly accessible, available in six languages. Data management support was provided by ISARIC where requested. ISARIC also hosted a COVID-19 data platform where investigators and public health agencies from 82 countries shared and collaboratively analysed data from more than 950,000 individual patient records collected using the forms. These accumulated data were used to plan health resources, design interventional research studies, support medicine licensing applications, improve clinical care, and inform public health policy by identifying risk factors for severe illness [22,23,24,25,26,27,28]. The scale of this global effort represents a significant milestone in global pandemic collaboration, meriting in-depth analysis to learn from this experience and improve approaches for future outbreaks.

ISARIC and WHO continue to collaborate on the development of CRFs to characterise emerging infectious diseases. To date, the most commonly method used to design the ISARIC-WHO forms and similar CRFs relies on a review of the available evidence and an expert consensus process [20,29,30,31,32,33]. While this approach is valuable, the resulting forms are not commonly evaluated post hoc to determine and improve their performance. For the purpose of informing a more efficient design of forms for future outbreaks, we analysed the use and completion of the ISARIC-WHO COVID-19 CRF. The objective of the analysis was to assess the utilisation of each data field by identifying which fields were included or omitted across the locally adapted and/or implemented CRFs and to determine which fields were most reliably completed with patient data. Using COVID-19 patient data submitted to the ISARIC platform, the field utilisation in data from patients with severe disease was compared to the field utilisation in all patients. Understanding the patterns of field selection and entry provides a data-driven approach to refining the ISARIC-WHO form to optimise data collection strategies for other emerging health threats. The results of this analysis will inform how ISARIC, WHO, and outbreak research responders can structure future CRFs to streamline data capture for priority research questions.

2. Materials and Methods

2.1. Data Collection

The ISARIC-WHO COVID-19 CRF and ISARIC-WHO Clinical Characterisation Protocol were designed, adapted, and implemented for prospective data collection on hospitalised patients suspected or confirmed to have COVID-19. Recruitment strategies were defined at a site or national level, with suggested approaches to limiting recruitment bias offered by ISARIC (see Appendix A and Appendix B). The protocol, informed consent forms, and CRFs are available on the ISARIC website [34]. Decisions regarding which forms to use and how to adapt them were made at an institutional or national level.

Pseudonymised individual patient-level data collected between 24 January 2020 and 20 March 2023 were shared with the ISARIC COVID-19 Data Platform for the purpose of collaborative analysis and evidence generation. The list of contributing sites and individuals is available on the ISARIC website (www.isaric.org accessed on 1 July 2024). Detailed methods of data collection and curation have been described previously [22]. Data that were not mapped to the central database were not included in this analysis. Data collection sites implemented data quality checks according to locally available resources. ISARIC verified data submitted to the COVID-19 Data Platform for missing and expected values and queried discrepancies with the sites. Data have been analysed extensively in collaborative studies by the ISARIC partners, driving iterative review and the validation of all datasets by those who generated the data.

Severe patients were defined as those with any of the following: admission to an intensive or high-dependency care unit; and treatment with inotropes, vasopressors, high-flow nasal cannulas, invasive mechanical ventilation, or non-invasive mechanical ventilation. The total, median, and interquartile ranges were determined for the number of sites, total patients, severe patients, and number of collection months of each institutional dataset, and across all datasets.

2.2. Description of CRF Sections and Fields

The ISARIC-WHO COVID-19 CRF has 243 fields divided across those collected at hospital admission (103 fields), ‘daily’ fields collected routinely each day during hospitalisation (63 fields), ‘summary’ fields collected on key events throughout hospitalisation (71 fields), and ‘outcome’ fields collected at death or discharge (6 fields) (Figure 1). The CRF fields are divided into groups based on the types of information they contain. Admission field groups include demographics, vital signs, signs and symptoms, comorbidities, pre-admission medications, and laboratory tests. Daily field groups include vitals and assessments, interventions, and laboratory tests. Diagnostics, interventions, and complications comprise the summary field groups. Outcome fields consist of outcome status (e.g., death, discharge, hospital transfer, and date) and outcome health (e.g., delayed discharge and ongoing care). The CRF, colour-coded with the results of this study, is available in the Supplementary Materials.

2.3. Evaluation of Field Inclusion and Completion

All fields on the ISARIC-WHO COVID-19 CRF (version 8DEC2021) were evaluated in this analysis, except for those added more than 12 months after the launch of the CRF (e.g., vaccination and virus variants), fields relevant to a particular subgroup of the population (e.g., infants), and fields that were not mapped to the central database (e.g., imaging). These fields were omitted from this analysis due to the absence of data in most patient records. No imputations were made for missing data.

The inclusion of each field from the original version into the versions used by individual institutions was at the discretion of each institution. Fields were considered included in an institutional dataset if the field featured within the CRF or if any data on that field were available in the dataset. The number and percentage of fields included in each dataset were calculated for each section of the CRF, as per Equation (1). The mean (see Equation (2)) and standard deviation of the number of fields included in each group of fields and across all datasets were determined. The results were visualised in a heatmap.

$$field inclusion=( {\displaystyle \frac{number of institutional datasets that included the field}{number of institutional datasets}})\times 100$$

(1)

$$mean field group inclusion={\displaystyle \frac{(field 1 inclusion+field 2 inclusion+\dots field n inclusion)}{number of fields in the group}}$$

(2)

For each field included by each site, the number of unique patients with data recorded in the field was divided by the total number of patients (Equation (3)). This was averaged across all datasets that included the field to establish each mean field completion across all data (Equation (4)). Where fields were dependent on other information (e.g., if pregnant: gestational age in weeks was recorded), the denominator of the completion proportion calculation was the total number of patients for whom the dependent information was true. Fields on the ‘daily’ section of the CRF were considered completed if they contained data for at least one day during admission. The mean field group completion was calculated across the datasets as per Equation (5).

$$field completion dataset 1=( {\displaystyle \frac{number of dataset 1 patients with data in the field}{number of dataset 1 patients }} )\times 100$$

(3)

$$mean field completion={\displaystyle \frac{(field completion dataset 1+field completion dataset 2+\dots field completion dataset n)}{number of datasets that include the field}}$$

(4)

$$mean field group completion={\displaystyle \frac{(mean field 1 completion+mean field 2 completion+\dots mean field n completion)}{number of fields in the group}}$$

(5)

2.4. Data Utilisation

Utilisation was quantified by calculating the Euclidean distance from an ideal point of (100, 100), representing the best case scenario of 100% completion proportion and 100% inclusion proportion. A greater distance from the ideal point indicates a reduced utilisation of a field, reflecting either less inclusion across sites, lower completeness, or both. Distances, hereafter called utilisation distances, for each field were calculated with all patient data using the mean proportion completed as the x coordinate and the mean proportion included as the y coordinate for each field (Equation (6)). Utilisation distances were normalised to a maximum of 100 by division with the highest distance, as per Equation (7), enabling comparison across fields.

$$\begin{gathered} individual field utilisation distance=\sqrt{{(100-x)}^2+{(100-y)}^2} \\ x=\mathit{mean} \mathit{field} \mathit{completion}, y=\mathit{field} \mathit{inclusion} \end{gathered}$$

(6)

$$normalised utilisation distance=\left( {\displaystyle \frac{field utilisation distance}{ \sqrt{{100}^2+{100}^2}}}\right)\times 100$$

(7)

Calculations were repeated using data from severe patients only. The utilisation difference between all patients and the severe cohort was determined. Results were represented in a scatter plot.

Statistical analyses were performed using Python programming language (version 3.8.8), with the pandas library (version 1.2.4) [35,36].

3. Results

A total of 1886 hospitals belonging to 42 institutions, networks, or public health agencies contributed data to ISARIC’s COVID-19 data platform. Clinical characterisation data on 950,064 suspected or confirmed COVID-19 patients from 82 countries were included (Figure 2). Datasets varied in size and scope. This range spanned 19 single-site collection efforts, a multinational network of 258 sites across 44 countries, and a national clinical surveillance program across 627 sites. Data collection began before the peak case numbers in any participating country, with a median start date of 17 February 2020, and lasted for more than a year in most sites (median 16.8 months, IQR 7.6–28.1 months of data collection) (

Table 1. Site and patient distribution across 42 institutional datasets included in the ISARIC COVID-19 Data Platform.

	Hospitals	Countries	All Patients	Severe Patients *	Data Collection Start Date	Months of Data Collection
Total	1886	82	950,064	256,529		725.7
Range	1–627	1–44	30–494,547	0–122,754	24/01/2020–17/03/2021	1.0–38.0
Median	1	1	1014.5	107	17/02/2020	16.8
IQR	1.0–28.0	1.0–1.0	182.8–4485.5	22.5–1147.8		7.6–28.1

* Severe patients are a subset of ‘All patients’.

and Table S1 in the Supplementary Materials). Across participating sites, 2155 staff were recorded as contributors to this data collection in the ISARIC database.

The proportion of severe patients included in each dataset ranged from 0% to 100% as datasets were collected in disparate settings, such as a district health centre in Maferenya, Guinea, and the intensive care unit of an academic medical centre in Karachi, Pakistan.

3.1. Field Group Inclusion

Out of 243 fields, a mean of 129.6 fields (SD 64.4) were included in the CRFs implemented across the institutions (

Table 2. Utilisation distance of data field groups in all and severe patient cohorts.

Collection Time	Field Group	All Patient Data			Severe Patient Data			Difference
		Utilisation Distance Mean (SD)	Completion % Mean (SD)	Inclusion % Mean (SD)	Utilisation Distance Mean (SD)	Completion % Mean (SD)	Inclusion % Mean (SD)	Utilisation Distance Delta (SD)
Outcome	Outcome status	8.0 (3.3)	89.8 (3.5)	95.2 (3.4)	3.7 (2.4)	95.7 (2.1)	97.5 (3.5)	4.3 (0.8)
Admission	Demographics	16.3 (14.0)	91.6 (8.2)	79.1 (18.7)	16.3 (14.0)	91.6 (8.2)	79.1 (18.7)	0.0 (0.0)
Admission	Comorbidities	22.9 (13.8)	79.0 (16.1)	77.9 (15.8)	22.9 (13.7)	83.2 (15.8)	74.1 (15.1)	0.0 (0.1)
Admission	Signs and symptoms	27.2 (14.4)	78.4 (10.7)	68.9 (18.6)	27.1 (13.3)	82.3 (7.5)	66.7 (18.6)	0.0 (1.1)
Admission	Vital signs	30.6 (10.8)	69.1 (13.5)	70.3 (9.3)	28.5 (11.4)	75.0 (14.5)	69.0 (10.0)	2.0 (−0.7)
Summary	Complications	41.8 (12.5)	65.5 (12.9)	52.8 (14.9)	37.5 (13.4)	78.1 (14.6)	52.4 (14.9)	4.3 (−0.9)
Admission	Pre-admission medication	47.3 (3.8)	70.5 (12.2)	40.5 (0.0)	46.5 (0.6)	79.8 (12.7)	38.1 (3.4)	0.8 (3.3)
Daily	Vitals and assessments	51.6 (12.3)	64.5 (23.0)	39.6 (13.9)	47.5 (11.3)	76.3 (18.3)	39.2 (14.2)	4.0 (1.0)
Summary	Diagnostics	52.8 (25.0)	54.0 (24.8)	42.3 (28.2)	52.0 (24.1)	59.1 (24.9)	40.1 (26.4)	0.8 (0.9)
Daily	Interventions	53.1 (14.3)	59.1 (24.5)	39.0 (9.2)	45.3 (8.0)	82.5 (11.4)	39.1 (9.7)	7.8 (6.4)
Summary	Interventions	55.0 (21.5)	61.2 (28.2)	35.5 (23.1)	52.5 (21.6)	67.6 (26.9)	35.8 (23.9)	2.5 (−0.2)
Admission	Lab tests	56.5 (12.0)	37.0 (15.1)	51.1 (9.8)	52.7 (12.6)	44.9 (16.9)	50.4 (9.4)	3.8 (−0.6)
Daily	Lab tests	58.4 (9.1)	39.3 (12.2)	44.3 (7.4)	52.1 (9.7)	52.6 (13.9)	44.1 (7.3)	6.3 (−0.6)
Outcome	Outcome health	78.8 (38.6)	19.9 (37.2)	22.6 (40.5)	78.3 (39.2)	20.5 (37.6)	23.1 (41.2)	0.5 (−0.6)
All	All	43.9 (21.0)	61.6 (24.3)	53.3 (22.4)	41.0 (19.9)	70.2 (22.7)	52.4 (22.0)	3.0 (1.2)

The mean proportion of datasets that included the field and completed the field is expressed in %. The mean utilisation distance expresses the mean distance from 100% inclusion and 100% completion of all fields in a given group. Delta is the mean field group utilisation distance in all patients minus the mean field group utilisation distance in severe patients.

). Fields collected at outcome and hospital admission were most often included. Outcome status fields had the highest mean field group inclusion rates (1.9/2, 95.2%), followed by the admission group fields of demographics (10.3/13, 79.1%), comorbidities (16.4/21, 77.9%), vital signs (7.7/11, 70.3%), and signs and symptoms (20.0/29, 68.9%). Other fields were less often included on the institutional CRFs. On average, laboratory test fields were more often included on admission forms than daily forms (51.1% vs. 44.3%) and intervention fields were more often included on daily forms than summary forms (39.0% vs. 35.5%). A total of 13/42 datasets (31.0%) that collected summary intervention fields collected only 1 or 0 daily intervention fields. Outcome health fields had the lowest mean inclusion of 0.9/4 (22.6%) (Figure 3).

Some field groups were omitted completely from several datasets. Pre-admission medications fields did not appear in 24/42 (57.1%) of databases. A total of 11/42 (26.2%) datasets had no daily data collection fields. Data for field group inclusion of individual datasets are available in Table S2.

3.2. Field Group Completion

The ranking of completion proportion by data field group was similar to that of the inclusion proportion. Demographics and outcome status had the highest mean completion values of 91.6% and 89.8%, respectively. The remaining admission field groups were the next highest, ranging from 79.0% completion of comorbidities fields to 69.1% of vital signs fields. Outcome health fields had the lowest mean completion (19.9%), followed by laboratory test fields at admission (37.0%) and daily (39.3%) (Table 2).

There was no appreciable difference in field inclusion between all patients and severe patients (mean decrease of 0.9% inclusion), nor in the completion of the most highly included field groups of demographic and outcome status data. However, the mean completion was higher in severe patients for all other field groups. Daily data were more often entered for severe patients, resulting in the largest increase in completion; intervention, laboratory tests, and vitals and assessment mean field group completion increased by 23.4%, 13.3%, and 11.8%, respectively. Complication fields had a mean increase of 12.6% more completion in severe patients (Table 2).

3.3. Field Group Utilisation

The utilisation distance, assigned to express proximity to a point of 100% inclusion and completion, followed the field group trends of its components, with lower scores demonstrating a higher utilisation. The mean utilisation distance across all 243 data fields was 43.9 (SD 21.0) in all patients and 41.0 (SD 19.9) in severe patients. Field groups with the lowest mean utilisation distance in all patients were outcome status (8.0, SD 3.3), demographics (16.3, SD 0.14), comorbidities (22.9, SD 13.8), signs and symptoms (27.2, SD 14.4), and vital signs (30.6, SD 10.8). Summary comorbidities had a mean utilisation distance of 41.8 (SD 12.5). Field groups collected daily were all higher than the mean utilisation distance, as were admission laboratory tests, summary diagnostics and interventions, and outcome health fields in all patients (Table 2). This ranking of field group utilisation distance was consistent in severe patients.

3.4. Individual Field Utilisation

The utilisation distances of individual fields highlight the variability of results within a group and identify specific fields that impact group utilisation. Subject IDs were assigned to all patients and were therefore included and fully completed in all CRFs (utilisation distance 0). Country was defined in the metadata of each dataset and was therefore included and complete for 100% of patients (utilisation distance 0). The next lowest utilisation distance in both cohorts included variables for sex (1.7), age (3.2), admission date (3.7), and outcome (5.7 in all patients and 2.0 in severe patients). Pathogen test results (8.7 in all and 9.3 in severe for pathogen type, and 9.7 in all and 10.6 in severe for test result) were additionally in the lowest utilisation distances. Other fields in the low (green) range of utilisation distances included comorbidities, signs and symptoms at admission, vital signs at admission, and additional outcome information. The central (yellow) range of utilisation distances were dominated by complications, interventions, vitals and assessments, and laboratory tests. The highest (red) of utilisation distances included fields for less common laboratory or diagnostic tests, outcome health, and details (type or duration) of interventions. Fields that capture features of severe illness feature more in the higher range of utilisation distances within each group (Figure 4).

The utilisation distance was lower in the severe patient cohort than in all patients, with a mean decrease in utilisation distance of 3.0 (SD 1.2) across all fields. Decreases were driven by higher completion proportions in the severe population. Daily interventions, daily laboratory tests, summary complications, and outcome status field groups had the largest decreases in utilisation distance (7.8, 6.3, 4.3, and 4.3, respectively). Individual fields associated with critical care were more likely to be included and/or completed in the severe patient cohort than in all patients. The largest decreases in utilisation distance were in fields for treatments targeting severe disease: extracorporeal membrane oxygenation type (30.0), dopamine treatment (19.4), non-invasive ventilation type (20.3) and duration (10.6), and high-flow nasal cannula duration (9.5). Examples of individual utilisation distances are presented in

Table 3. Examples of utilisation distance of individual fields in all and severe patient cohorts.

Collection Time	Field Group	Field	Utilisation Distance, All Patients (x, y)	Utilisation Distance, Severe Patients (x, y)	Delta	Utilisation Colour Group (Severe)
Admission	Demographics	Sex	1.7 (99.9, 97.6)	1.7 (99.9, 97.6)	0
Daily	Vitals and assessments	O2 saturation	42.3 (71.2, 47.6)	38.7 (84.1, 47.6)	3.6
Outcome	Outcome status	Outcome	5.7 (92.3, 97.6)	2.0 (97.2, 100)	3.7
Admission	Lab tests	INR	58.5 (30.8, 54.8)	54.6 (39.2, 52.4)	3.8
Summary	Complications	Bacteraemia	32.4 (71.4, 64.3)	27.3 (85.3, 64.3)	5.0
Daily	Lab tests	Troponin	73.4 (18.4, 35.7)	67.8 (28.8, 35.7)	5.6
Summary	Complications	ARDS	30.3 (68.0, 71.4)	24.0 (81.6, 71.4)	6.3
Daily	Lab tests	ALT	52.1 (45.9, 50.0)	45.8 (58.8, 50.0)	6.3
Daily	Vitals and assessments	Glasgow coma score	51.9 (48.5, 47.6)	45.2 (63.5, 47.6)	6.8
Daily	Lab tests	PT	70.7 (21.4, 38.1)	63.3 (35.4, 38.1)	7.5
Daily	Lab tests	Lactate	63.7 (28.5, 45.2)	55.0 (44.8, 45.2)	8.7
Daily	Vitals and assessments	PaO2 sample type	64.9 (34.4, 35.7)	56.0 (53.6, 35.7)	8.9
Summary	Interventions	High-flow nasal cannula duration	81.1 (26.6, 11.9)	71.5 (50.3, 11.9)	9.5
Summary	Interventions	Non-invasive vent duration	74.3 (10.3, 45.2)	63.7 (28.5, 45.2)	10.6
Daily	Interventions	Dopamine	64.9 (34.5, 35.7)	45.5 (97.1, 35.7)	19.4
Daily	Interventions	ECLS/ECMO type	90.9 (2.1, 16.7)	60.9 (78.2, 16.7)	30.0

Fields listed are a random sample. Delta is the utilisation distance in all patients minus the utilisation distance in severe patients. Utilisation distance is a function of the mean field completion (x) and field inclusion (y). Utilisation colour groups in severe patients: light green indicates fields with utilisation distances that decreased to below 30 in severe patients; orange indicates fields with utilisation distances that decreased to below 70 in severe patients; dark green indicates fields with utilisation distances lower than 30 in all patients and severe patients; yellow indicates those from 30 to 70; and red indicates those greater than 70.

, with the utilisation colouring matching that in Figure 4. Utilisation scores for all fields are available in Table S3, in the Supplementary Materials.

4. Discussion

The implementation of the ISARIC-WHO COVID-19 CRF, or its derivatives, has been evaluated in 42 distinct data collection initiatives across 1886 hospitals. Despite their differences in geography, context, data collection periods, and volume of data collection, consistent patterns of data field inclusion and completion are observed across these institutions.

4.1. Field Inclusion

Of the 243 fields included on the ISARIC-WHO COVID-19 form, the mean number of fields included in CRFs adapted and implemented by ISARIC partners is 129.6 (53.3%). The low mean field inclusion resulted in high proportions of missing data across the aggregate data, particularly in the outcome health, laboratory, and intervention field groups. This restricted the populations that could be included in cross-institutional analyses, limiting the power and generalisability of the results to inform clinical and public health decision making.

Data collection, and the adaptation and/or implementation of the CRF used, began early in the outbreak. The majority of sites had already initiated data collection before the WHO declared COVID-19 a pandemic on 11 March 2020 [37]. Therefore, decisions about which variables to include in the institutional forms were made before the magnitude of COVID-19’s impact was known. Motivations to reduce the number of fields included in the institutional forms may have included limitations of infrastructure, patient management, data flow, human resources, and staff motivation.

The low inclusion of intervention fields (39.0% and 35.5% on the daily and summary forms, respectively) may be due to a lack of equipment for higher-level care, such as different types of ventilation, dialysis, or extracorporeal membrane oxygenation [38]. Several participating sites in low-resource settings did not have access to these interventions. The omission of laboratory fields to align with patient management guidelines is a practical decision when tests, such as that for interleukin-6 (38.1% and 35.7% included in the admission and daily forms, respectively), are not part of standard care [39].

Data flow patterns limited the inclusion of fields in institutional data collection when the source of data used to complete the CRF did not cover all CRF fields. For example, when electronic health record data extraction is used to populate the CRF, the information required for some data fields may not be available in the records [40]. In other cases, it may not be possible to adapt pre-existing data systems such as patient registries to accommodate all data in the CRF. Some sites reported reducing fields that required clinical judgement as data collection was undertaken by data staff without clinical training. Other sites omitted fields that required data that were not available in medical records [41]. These factors are likely contributors to the low inclusion of fields on outcome health (3/4 fields had only 2.4% inclusion).

Completing and entering data for the entire ISARIC-WHO COVID-19 CRF was reported to take approximately 30 min per patient when inpatient stays are limited, and a local ethics committee waived the requirement for written informed consent [42]. Bandwidth limitations may extend this time for electronic data capture. During a developing public health emergency, human resource constraints due to caseloads and/or staff illness may lead to conservative research data collection approaches despite the need for evidence. Two nation-wide research programs that contributed to the ISARIC COVID-19 Data Platform illustrate this. In the United Kingdom, the ISARIC4C study included 75.7% of the CRF fields to collect data on >300,000 patients in 323 hospitals under the direction of national health authorities who mobilised dedicated research staff to collect data in all hospitals [17,43]. In South Africa, the National Institute of Communicable Diseases conducted a nationwide hospital surveillance and research study across 627 hospitals. Without dedicated research staff available, the decision was made to reduce the ISARIC-WHO CRF down to a feasible 20.6% of the original fields, which were collected for >490,000 patients [19,44]. These findings highlight the value of dedicated research staff who can be seconded for outbreak research response when needed.

Though differing operational limitations define institutional field inclusions and exclusions, the overall inclusion trend indicates alignment in research priorities across the institutions. Outcome status fields and key admission fields (demographics, comorbidities, vital signs, and signs and symptoms) had the highest inclusion rates across the institutions (68.9–95.2%). The focus on these time points is supported by scientific justification from several sources, including Harris et al.’s 2020 review of key early outbreak research questions for SARS-CoV and MERS-CoV [45], and Rojek et al.’s list of minimal observational data needed to design clinical trials for high-priority pathogens [1]. These highlight clinical presentation, risk factors for death and severe illness, optimal diagnostics, case defining features, risk factors for infection, and outcomes as research priorities. All of these can be addressed by information collected at admission and/or outcome. The WHO’s 2020 Strategic Preparedness and Response Plan for the Novel Coronavirus echoes this inventory of priorities with a research focus on disease severity and identifying at-risk groups [46].

4.2. Field Completion

Unlike the institutional decisions that drove inclusion decisions, the completion of data fields relied on individual clinical and research staff. The completion of fields is subject to operational barriers when institutional decisions cannot account for all circumstances. In addition, independent decisions can be made by the individuals responsible for data collection. Though pre-admission medication fields were included on 61.9% of forms, they were completed just 40.5% of the time. This indicates a likely absence of information on the data source, and/or limitations of staff time to complete all fields.

Fields related to outcome health had consistently low completion rates across institutions (19.9% mean completion for all patients). This information regarding the health needs of the patient post-discharge may not have been assessed or available at the point of outcome, or it may have been perceived as low value or outside of the responsibilities of those collecting these data. Though many studies were later executed to understand long COVID, the absence of these data in ISARIC’s pioneering characterisation work prevented the opportunity for earlier detection of sequela. Laboratory tests had the next lowest completion rates (37.0% and 39.3% for admission and daily data in all patients, respectively), likely due to many parameters not being measured as a part of clinical care. This is supported by the increase in completion for severe patients (44.9% and 52.6% at admission and daily, respectively), whose health complications would warrant additional laboratory monitoring.

4.3. Adjusting for Severity

Increased data completion for severe patients accounted for a higher field utilisation in most field groups. Additional data may enable an understanding of the mechanisms of severe disease and inform effective treatment. However, without a systematic severity-based collection strategy, individual staff may apply ad hoc approaches based on perceived relevance of data in higher-level care and/or increased availability of information due to more frequent monitoring. Prospective planning of research questions and the data required to address them is needed to promote efficiency in evidence generation.

The additional data required to understand severe disease should be collected only in the relevant patients. The consistency of field inclusion between all patients and severe patients reflects institutional-level decisions to use the same CRF across all levels of severity. The practicality of this decision highlights the importance of CRFs that can adapt to different data requirements where relevant to the severity of illness.

4.4. Limitations

The data used in this analysis were collected from diverse institutions with varying levels of resources and capacities. This diversity, while providing a broad perspective, does not represent all settings in which research on emerging infectious diseases is important. The ISARIC collaboration relied on voluntary participation and data sharing, which may introduce selection bias for institutions with interest, resources, and an enabling regulatory framework for these activities. This bias may predispose the inclusion and completion results of participating institutions to be higher than a true average of all contexts. When planning data collection that requires robust representation, it is important to make provisions to include health care facilities that lack research resources. The interpretation of clinical evidence generated only from sites with research resources should also consider potential biases in patient outcomes.

Differences over time or between different settings were not explored in this analysis. Both may have been drivers of change in completion rates. Additionally, as the data are all from patients with COVID-19, differences in research priorities for different emerging syndromes may require the collection of data on different fields and/or events. These should be considered based on the context and state of knowledge and can consider how the results of this study may be applied.

While we identified fields and field groups with lower utilisation based on inclusion and completion rates, these metrics do not fully capture the clinical significance or research value of individual data fields. Some fields may be underreported yet hold critical importance for specific populations, research questions, clinical decisions, or health policy.

4.5. Towards Optimising Clinical Epidemiology in Disease Outbreaks

Ensuring that research resources are used efficiently is important to maximise the improvement in patient outcomes in all diseases. In the context of emerging infectious diseases with limited evidence on effective interventions, this efficiency is particularly important to inform a rapid and reliable response that reduces the impact of the outbreak. Additionally, by minimising the time required for data collection, health care workers can dedicate more attention to providing front-line patient care.

The results of this study inform several key efficiency-focused improvements to CRFs for the clinical characterisation of diseases in future outbreaks. Guided by the utilisation of each field, planned changes for future ISARIC-WHO CRFs include the modularisation of contents, operational streamlining in field selection, and iterative review based on accruing evidence. These approaches will enhance the clinical and public health values of the data collected and improve data capture efficiency.

CRFs should focus on locally prioritised research and public health questions, designed with incremental modules that can be completed or omitted as resources allow. Sites with limited data collection resources can complete essential fields capturing key case-defining features, the spectrum of clinical presentation, risk factors, laboratory diagnosis, and outcomes. Sites with more resources can include additional fields for a more comprehensive understanding of the disease spectrum. A third module of daily data collection can be available for capable sites, generating detailed evidence on the disease course, patient journey, and specific populations. Fields relevant only to research questions on severe disease or patients in critical condition should be in a standalone module, used only for patients who achieve a threshold of severity.

This tiered approach ensures all sites focus their data collection capacity on the knowledge gaps most relevant to their patients. Sites may adjust the number of modules based on caseload, resource availability, or other factors as they change during an outbreak. It is important to understand and account for potential bias in any approach that focuses on well-resourced centres.

Fields that cannot be completed at a hospital should be excluded and documented to support the understanding of missing values. This allows researchers to make informed decisions about appropriate methods for handling the missing information, address potential biases, and interpret the results.

To ensure the relevance and comprehensiveness of the CRF, dynamic updates should be made based on the continuous review of research priorities and data utilisation monitoring employing the methods described in this paper. By maintaining flexibility and responsiveness in data collection processes, CRFs can support efficient clinical characterisation and enhance the quality and usability of collected data during outbreaks.

5. Conclusions

A data-driven approach to designing CRFs for the clinical characterisation of emerging infectious diseases can enhance data collection efficiency to inform patient care and public health response. The selection of data collection fields should always be tailored to prioritised research objectives, the outbreak dynamics, and the context of local health systems. The availability of tiered, standardised, template CRFs that optimise the inclusion of relevant data fields can accelerate the generation of critical evidence. The adoption of standard data variables and modular options additionally promotes the alignment of global data collection efforts, offering further opportunity to reduce time to evidence through aggregate analysis that impacts policy. These principles can increase the efficiency of evidence generation in many types of exploratory data collection. ISARIC and WHO will continue to develop CRFs for emerging infectious diseases, based on locally and globally defined research priorities and the practical insights gained from their actual use.