**3. Results**

#### *3.1. Comparison of Test Procedures*

Although the current WHO test procedures recommend a single DC assay to detect resistance [3], the WHO tube bioassay method initially recommended a concentration response experiment [15]. Field-caught blood-fed females were used, with 15–25 mosquitoes per test unit using a series of four concentrations, which should lie on a range giving 0–100% mortality with four replicates per concentration for a total of 200 mosquitoes per test concentration. If a population of mosquitoes was highly resistant, the exposure time was increased by 1 h until significant mortality was seen. This method continued to be recommended until 1970, when the method changed from recommending four concentrations to only two concentrations, with the lowest concentration to be tested first with a range of exposure times [16]. At a WHO meeting in 1976, this was changed again to a single concentration known as the discriminating concentration [17]. These updates were made in order to simplify the bioassay to fewer test concentrations for the growing list of insecticides which required resistance monitoring.

The test kit itself was initially eight exposure tubes marked with a red dot, two control tubes marked with a green dot, and ten holding-tubes also marked with a green dot (see Figure 2). Moreover, it was specified that the impregnated papers could initially be reused up to 20 times, and the test kit had to be oriented with the mesh screen facing up during exposure.

Several meetings were held between the years 1958 and 1992 to discuss the changes to the methodology to address increased resistance to insecticides and to add new insecticide classes (See Table 2 for a list of meeting reports) [2,15–20]. The test procedures were then updated in 1998 following a multicenter study which recommended DCs for five pyrethroid insecticides [21]. This update also included some methodological changes. Single discriminating concentrations were provided for both the newly added pyrethroid insecticides and the organochlorines, organophosphates, and carbamates. Minor adjustments were made

to the test kit itself to reduce it from a 20-test unit kits to a 12-test unit kit consisting of five exposure tubes marked with red dot, two control tubes marked with a green dot, and five holding-tubes also marked with a green dot. Testing a minimum of 100 mosquitoes (4–5 replicates of 20–25 mosquitoes) per concentration was recommended. Mosquitoes for testing were now required to be 1–3-day-old non-blood-fed females. These mosquitoes were either F1 progeny from larval collections or field-caught mosquitoes. The temperature range of 25 ± 2 ◦C and 70–80% RH was specified. Insecticide-treated papers were only able to be used 5 times, as opposed to the previously recommended 20. The vertical orientation of the test tubes during performance of the bioassay was further justified in these test procedures, as horizontal positioning avoids the knockdown and recovery of mosquitoes, since knocked down mosquitoes would lie on treated paper instead of the untreated mesh-end of the test unit and so still be exposed to the insecticide. This would increase the exposure of the mosquito, and the exposure route may not be through the tarsi of the mosquito [21].

The WHO test procedures were then updated again in the 2006 "Guidelines for testing mosquito adulticides for residual indoor spraying and treatment of mosquito nets". Little changed between the 1998 version of the test procedures and this version; the recommended humidity changed from 70–80% RH to 80 ± 10%, and 2–5 day old mosquitoes were specified instead of the previous 1–3 day old [22]. Then, in 2013, the "Test procedures for insecticide resistance monitoring in malaria vector mosquitoes" was published. Minor adjustments were made to the test kit itself; the new 12-test-unit kit consisted of four exposure tubes marked with a red dot, two control tubes marked with a yellow dot, and six holding-tubes also marked with a green dot. At least 120–150 active 3–5-day old female mosquitoes were recommended to be exposed in batches of 20–25, ideally with at least 100 per insecticide and 50 as controls [23].

The most recent update to the test procedures came in 2016 [25]. These procedures aimed to provide a stronger focus on producing operationally meaningful data and so introduced resistance intensity (RI) assay testing (using 5× and 10× the pyrethroid DC) and pyrethroid-PBO synergist bioassays as additional testing alongside the standard WHO insecticide susceptibility bioassay. Again, slight changes were made to the WHO tube bioassay protocol. The temperature and humidity changed to 27 ± 2 ◦C and 75 ± 10% RH, and it was recommended that the test units be "placed in an area of reduced lighting or covered with cardboard discs". This was supposed to reduce the light intensity and discourage mosquitoes resting on the mesh. There was also an additional piece of WHO documentation in the 2016 "Monitoring and managing insecticide resistance in *Aedes* mosquito populations Interim guidance for entomologists", which was published as part of the response to the Zika epidemic. However, there were no methodological differences in performance of the bioassay from the previously published 2016 procedures [25]. The same methods are thus recommended for *Aedes* spp. as for *Anopheles* species.

**Table 2.** Summary of the review of historic versions of the World Health Organization (WHO) tube bioassay guidelines. \* Initial baseline dose



**Figure 2.** Original World Health Organization (WHO) tube method as outlined in the "8th Report of the Expert Committee on Insecticides" [24], reproduced with permission of Rajpal Singh Yadav, WHO. (**A**) Collect test mosquitoes using a mouth aspirator. (**B**) Mosquitoes should be collected in batches of no more than 10. (**C**) Test mosquitoes are gently transferred to the holding-tubes until they number 20–25 per tube. (**D**) The exposure tube is attached, and the slide is opened. Mosquitoes are then gently blown from the holding-tube to the exposure tube. The holding-tube is detached and set aside (**E**) The exposure tubes are left standing upright for 1 h during the exposure. (**F**) Mosquitoes are transferred back to the holding-tube by reversing the process described in C. The holding-tube is set upright, and a pad of wet cotton wool is placed on top. Tubes are held for 24 h, at which point mortality counts are made.

The criteria for scoring knockdown and mortality in this bioassay have remained unchanged. However, there is room for interpretation around what is or is not a knocked down mosquito. When testing pyrethroids with adult mosquitoes, it is common to see surviving individuals with several legs missing. These mosquitoes are still technically alive and able to fly but have clearly been impacted by the exposure. To take this into account, Hougard et al. assessed "functional mortality" alongside normal mortality scor-

ing (dead mosquitoes only). Functional mortality was defined as "including surviving mosquitoes with three legs or fewer", as it is assumed that mosquitoes with three legs or fewer would not survive in the field. From this study, considering functional mortality provided additional information as well as a better estimate of the overall killing effect of a pyrethroid insecticide [26]. However, Isaacs et al. showed that insecticide-induced leg loss had no significant effect upon either the blood-feeding or egg-laying success of exposed mosquitoes. A non-significant reduction in blood-feeding success was seen with 1-legged insecticide-exposed mosquitoes, and, while their egg laying behavior appeared to be altered, the eggs laid were fertile and hatched to larvae. We conclude that studies of pyrethroid efficacy should not discount mosquitoes that survive insecticide exposure with fewer than six legs, as they may still be capable of biting humans, reproducing, and contributing to malaria transmission [27].

#### *3.2. Review of the Literature*

Only the 1998, 2006, 2013, and 2016 test procedures were referenced in the sampled publications, with the majority referencing either the 1998 or the 2013 test procedures. However, when comparing the publication date of a journal article and the publication dates of the test procedures referenced within, over half the publications were using test procedures that were between 3 and 18 years out of date (See Figure 3).

**Figure 3.** The number of years out of date the referenced guidelines were for a given publication in relation to the most recent guidelines available at the time of publication.

The test procedures have remained consistent since 1998 in outlining the number of mosquitoes per test unit as 20–25. When looking at the number of mosquitoes per test unit used for testing in the published literature (see Figure 4), approximately 90% were within the WHO range. Those that lay outside the range tended to use between 10 and 15 mosquitoes; these were often field studies, and so this was likely due to the limited availability of mosquitoes in the field. This was also mirrored in the number of mosquitoes used per treatment. The WHO recommend 100 per treatment, but again field studies often used less than this, which again was probably because of mosquito availability. Studies that showed numbers of mosquitoes per treatment larger than 250 were often pooled from multiple sites or multiple rounds of testing. However, 44% of the publications sampled reported "20–25" mosquitoes instead of the actual numbers used per test unit, which shows that they followed the test procedures but does not provide accurate 'n' values for a given treatment. Several papers reported using mosquitoes in the range of "15–25", "10–15", or "10" per test unit. No justification for this deviation from the WHO test procedures for this bioassay is provided within the publication. However, it can be assumed that, due to these studies either using field-caught larvae reared to adults or F1 larvae of field-caught adults reared to adults for their bioassay testing, they would be limited in terms of total sample size and so reduced the number per test unit to increase technical replication.

**Figure 4.** The number of mosquitoes used in an individual tube for publications reviewed.

Since 2013, the WHO test procedures have recommended a minimum of 50 mosquitoes to be exposed to control papers in 2 batches of 25 each alongside the 100 required per treatment. This is often not reported in the literature, with around 80% of publications not reporting this information; however, this is unsurprising, as it is a more recent addition to the WHO test procedures.

The source of the exposure papers is often not reported, with nearly 45% of publications using terminology along the lines of "papers impregnated with insecticide were used"; however, it is unclear from this whether papers were made by the researchers themselves or purchased from Malaysia. Since 1993, the WHO have provided standardized insecticide papers from their site in Malaysia, and over 40% of publications stated that their exposure papers were sourced from there. The studies which did specify the source of papers as other than from the WHO either impregnated their own exposure papers, had them made up by a partner research institute, or purchased them from a center for disease control or other public health body.

The recommended mosquito age has changed several times throughout the different iterations of the test procedures. In 1998, 1–3 days was recommended, until this was updated in 2006 to 2–5 days and again in 2013 to 3–5 days. For publications referencing the 1998 test procedures, 85% used mosquitoes older than recommended. For publications referencing the 2013 test procedures onwards, 23% used mosquitoes younger than recommended. So, 44% were using mosquitoes of the incorrect age for the test procedures they referenced (see Figure 5).

The sampled manuscripts described the results from a range of Anopheline and Culicine species, though the species were not always identified, as well as a large number of insecticides from different mode of action classes (detailed in Supplementary File S1). In instances where more than one publication tested the same combination of strain of mosquito and insecticide, we compared the data between the two publications. A total of 44% of publications used only a field strain and so data was not comparable. For the remaining publications, 38% included a susceptible *A. gambiae* (Kisumu), 13% used an unspecified laboratory strain, two publications used a susceptible *A. funestus* (FANG), and one publication used the susceptible *A. coluzzii* (N'gousso) as reference strain alongside the testing of field populations. The data for these susceptible reference strains agreed between

the publications; however, the mortality was often 100%, as the strain being tested was a susceptible laboratory strain. Three publications exposed resistant mosquito strains to discriminating concentrations to profile their resistance phenotype. Bagi et al. [28] and Williams et al. [12] both exposed Tiassalé 13 to 0.75% permethrin for 1 h; Bagi showed a 3.4% mortality 24 h post-exposure, whereas Williams et al. showed ~20% mortality when the strain was profiled in the years 2017 and 2019 [12,28]. Owusu et al. [29] also exposed Tiassalé 13 to 0.75% permethrin for 1 h and showed a mortality of 78.0%, whereas Williams et al. [12] showed approximately 5% mortality for the years 2017 and 2019.

**Figure 5.** The age of mosquitoes tested for publications reviewed.

#### *3.3. Experimental Investigation of Parameters*

The GLMM accounting for biological effect was used to generate the effect estimate for the two variables of interest. The Kisumu strain was much more susceptible to knockdown, assessed at 60 min, when there was a reduction in the number of mosquitoes per tube with a significant reduction in knockdown for tubes with 20 (OR = 0.42, *p* = 0.001, 95% CI = 0.26–0.69), 15 (OR = 0.35, *p* ≤ 0.001, 95% CI = 0.21–0.59), and 10 (OR = 0.2, *p* ≤ 0.001, 95% CI = 0.11–0.36) mosquitoes. This significant reduction was still found when evaluated again at 24hrs in the tubes of 10 and 20 mosquitoes; however, this was no longer present for the tubes containing 15 mosquitoes. For the Tiassalé 13 data, the 60 min assessment also found a significant reduction in tubes containing 10 mosquitoes (OR = 0.28, *p* = 0.004, 95% CI = 0.12–0.67); however, this effect was not discernable when evaluated again at 24 h (Figure 6, Appendix A).

An additional treatment was performed with 25 mosquitoes per test unit with a cardboard disc covering the top of the tube during exposure. No significant difference was detected for this alteration in the study protocol in either the Kisumu strain (OR = 0.83, *p* = 0.391, 95% CI = 0.53–1.28) or Tiassalé 13 at 24 h (OR = 1.27, *p* = 0.462, 95% CI = 0.67–2.39).

For the Kisumu strain, the 4–7-day-old and 6–9-day-old mosquitoes showed a significant increase in mortality at 60 min knockdown. However, only the 6–9-day-old mosquitoes maintained this statistical significance when assessed at 24 h (OR = 2.46, *p* ≤ 0.001, 95% CI = 1.69–3.59). The mortality assessment for the 2–5-day group's mortality increased from around 26 to 64% between assessment periods.

**Figure 6.** Bioassay data looking at the number of mosquitoes used in testing may impact the result of standardized bioassay testing for an *Anopheles gambiae* susceptible Kisumu strain and a resistant Tiassalé 13 strain. A total of 25 per tube (C) had the top of the tube covered during exposure, while 25 per tube (U) had the same conditions as the other tubes. Error bars equate to the 95% confidence intervals of the proportion.

This trend for older mosquitoes to show a greater susceptibility following exposure was also seen for the Tiassalé 13 strain (Figure 7) with the 6–9-day-old group showing an increase in mortality of ~9% points compared to the 2–5-day-old group at 60 min (OR = 2.99, *p* ≤ 0.001, 95% CI = 1.64–5.46), and both the 4–7-day- and 6–9-day-old groups showing increased mortality at 24 h (Figure 7).

**Figure 7.** Bioassay data looking at how age of mosquitoes used in testing may impact the result of standardized bioassay testing for a susceptible *Anopheles gambiae* Kisumu strain and a resistant Tiassalé 13 strain. Error bars equate to the 95% confidence intervals of the proportion.
