*3.1. Ethics*

A sound ethics framework must underpin the research studies, with an understanding that informed consent may be required from patients (this was deemed not to be needed in any of the four TB case studies) and the benefits and risks of any intervention may need careful assessment from a human rights perspective and the disease control programme itself [29]. In the four case studies, ethics approval was always sought, although in some cases, the National Scientific and Research Ethics Committee waived the need for a formal ethics review.

#### *3.2. Research Relevance and Prioritization at the Country Level*

The research must be relevant and a priority for the country and the disease control programme. The operational research studies in the four examples were all priorities for the national TB programmes. Programme directors were kept closely in touch with the data collection and analysis that occurred during the study and were all involved in the stakeholder meetings at the conclusion of the study. As such, the results were influential in shaping policy and practice on the ground and making a difference for the better.

Real-time operational research is beneficial in pandemic and epidemic situations where countries are desperate to maintain routine health services despite numerous challenges. The anticipated negative impact of COVID-19 on TB and HIV services in Kenya, Malawi and Zimbabwe was a case in point, where monthly instead of quarterly surveillance was welcomed and instituted by the disease control programmes to try and counteract the dramatic decreases in TB case detection and HIV testing. The data were also used to assess new strategies implemented to counteract the adverse effects of COVID-19 [24–26].

Another example is the Ebola Virus Disease (EVD) outbreak in Sierra Leone and Liberia in 2014. Operational research carried out after the EVD outbreak showed the adverse effects of EVD on the ability of the programmes to diagnose TB and provide HIV testing [30,31]. While this research provided useful information about what to do in the event of further EVD outbreaks, real-time operational research at the time of the 2014 EVD outbreak might have helped the programmes better weather the storm, as was shown for example with TB services in neighbouring Guinea [32].

#### *3.3. Research That Adheres to International Standards of the Conduct and Reporting of Research*

The research, whether this is designed as a cross-sectional study, a cohort study or a case-control study, must be conducted and reported according to international standard guidelines [33]. The majority of the operational research studies that focus on quantitative data are observational designs and should follow the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines [34]. Qualitative research studies should follow the Standards for Reporting Qualitative Research (SRQR) [35] or the consolidated criteria for reporting qualitative research (COREQ) guidelines [36]. The four case studies on TB followed STROBE guidelines. Adherence to these quantitative and qualitative guidelines ensures that the research is conducted and reported to international standards and adds further credibility to the research study findings.

### *3.4. Comparison Groups*

An important methodological consideration for real-time operational research is whether or not to have a comparison group. In the first case study on the misregistration of recurrent TB, the first published study had no comparison group. However, once guidelines had been developed and disseminated, the first study acted as a comparison for the second study to assess whether misregistration had decreased [8,10].

In the second case study, a package of counselling, HIV testing and adjunctive cotrimoxazole administered to a cohort of TB patients was compared with no package in a cohort of TB patients registered the previous year (defined as a historical control) [13]. The use of historical controls is acceptable, although it is important that the investigators clearly outline any differences in the cohort composition and/or standards of care of patients before and during the intervention that might affect the primary outcome.

Another way of using a control group is to select a concurrent comparison cohort against which to judge the intervention results. For example, a study was done in Malawi to assess TB treatment outcomes in a cohort of patients offered the package of HIV testing and adjunctive cotrimoxazole in Thyolo district in 2001 and results compared with the neighbouring Mulanje district during the same time where no such interventions were offered to registered TB patients [37]. There was a significant increase in treatment success and a significant decrease in mortality and other adverse outcomes in Thyolo than in Mulanje district, with findings comparable to those of the original study conducted in Thyolo district [13]. The use of concurrent controls is also acceptable, although care must be taken to match the situation of the intervention and comparison groups as close as possible.

Not all operational research studies need comparison groups. The third case study on screening TB patients for DM had no such group [21]. However, if the stakeholder committee had decided on two different screening methods, then a comparison study would have been needed.

#### *3.5. Timely Collection and Sharing of Data*

Consideration must be given to the timely collection, sharing and validation of data. The first two case studies from Malawi [8,10,13] and the third from India [21] used paperbased questionnaires and data collection forms for the primary data. All the investigators were in the country, and there was no problem with the sharing of data. The fourth case study, however, had investigators from several countries, and there was a need to regularly check and validate the data between the overall monitoring and evaluation coordinator based in India and the country coordinators based in Africa. Doing this with paper-based records was going to be difficult, if not impossible. In the Kenya, Malawi and Zimbabwe studies [24–26], an EpiCollect5 application (https://five.epicollect.net, accessed on 7 June 2021) was therefore used to collect the aggregate data and this was used in realtime to cross-check and validate data for the different variables and ensure that numbers added up. EpiCollect5 is a free application that can be downloaded to smartphones, and it was relatively easy to train all the data collectors in the country about how to use it.

#### *3.6. Dissemination and Getting Research into Policy and Practice*

In the four case studies, key implementers carried out the research, but the decisionmakers in the Ministry of Health were involved right from the start in terms of conceptualization, methodology, analysis and interpretation of the data and the writing and editing of the final manuscript. Taking a research project from protocol to publication acts as a form of quality control, demonstrating to investigators and key decision and policymakers that the science is of a good standard and the findings reliable for making decisions [38]. The publication is important and, particularly if Open Access journals are used, allows for the widespread dissemination to national and international audiences.

Dissemination, for example, at a stakeholders' meeting where decision and policymakers are present, is another essential part of sharing the findings and persuading people to adopt the recommended changes to policy and practice. In the first three case studies, interim stakeholder meetings with programme directors were held as the studies progressed, and importantly they were also held within a few weeks or months of the studies being completed leading to important national decisions being made long before the papers were published [10,15,23]. In the fourth case study, the data were put together each month as figures, tables and a narrative and sent as monthly reports to programme directors [24–26]. The monthly deadlines for submission were never missed, which allowed confidence to be built into the system, especially for trying out and monitoring new strategies.

#### *3.7. Research Capacity Building*

Skill and experience are needed to plan and conduct real-time operational research. Over the last 12 years, we have used the SORT IT (Structured Operational Research and Training Initiative) model to build capacity in programmatic or public health officers in low- and middle-income countries to undertake and publish operational research using routinely collected data [39]. This has been remarkably successful, resulting in many high-quality observational studies being undertaken and published in open access journals and adhering to STROBE guidelines [40].

SORT IT courses are structured with three one-week modules over a fixed relatively short period of 9–12 months, and as a result, most studies use routinely collected secondary data. More thought will be needed to adapt the SORT IT model to real-time operational research. This will require longer time periods between modules to ensure that (i) patientcentred ethics approvals are obtained in time, (ii) investigators are properly trained in the collection and validation of data and (iii) sample sizes are adequate to judge whether interventions are effective.

The fourth case example [24–26] also shows how capacity can be built on the ground in existing staff working on the front-line in health facilities. The Union central monitoring and evaluation coordinator trained all existing staff in Nairobi, Lilongwe and Harare about how to use EpiCollect5, useful knowledge not only for the study but also for future public health work.
