Comparison of cone bioassay estimates at two laboratories with different Anopheles mosquitoes for quality assurance of pyrethroid insecticide-treated nets

BACKGROUND: Quality assurance (QA) of insecticide-treated nets (ITNs) delivered to malaria-endemic countries is conducted by measuring physiochemical parameters, but not bioefficacy against malaria mosquitoes. This study explored utility of cone bioassays for pre-delivery QA of pyrethroid ITNs to test the assumption that cone bioassays are consistent across locations, mosquito strains, and laboratories. METHODS: Double-blinded bioassays were conducted on twenty unused pyrethroid ITNs of 4 brands (100 nets, 5 subsamples per net) that had been delivered for mass distribution in Papua New Guinea (PNG) having passed predelivery inspections. Cone bioassays were performed on the same net pieces following World Health Organization (WHO) guidelines at the PNG Institute of Medical Research (PNGIMR) using pyrethroid susceptible Anopheles farauti sensu stricto (s.s.) and at Ifakara Health Institute (IHI), Tanzania using pyrethroid susceptible Anopheles gambiae s.s. Additionally, WHO tunnel tests were conducted at IHI on ITNs that did not meet cone bioefficacy thresholds. Results from IHI and PNGIMR were compared using Spearman’s Rank correlation, Bland–Altman (BA) analysis and analysis of agreement. Literature review on the use of cone bioassays for unused pyrethroid ITNs testing was conducted. RESULTS: In cone bioassays, 13/20 nets (65%) at IHI and 8/20 (40%) at PNGIMR met WHO bioefficacy criteria. All nets met WHO bioefficacy criteria on combined cone/tunnel tests at IHI. Results from IHI and PNGIMR correlated on 60-min knockdown (KD60) (r(s) = 0.6,p = 0.002,n = 20) and 24-h mortality (M24) (r(s) = 0.9,p < 0.0001,n = 20) but BA showed systematic bias between the results. Of the 5 nets with discrepant result between IHI and PNGIMR, three had confidence intervals overlapping the 80% mortality threshold, with averages within 1–3% of the threshold. Including these as a pass, the agreement between the results to predict ITN failure was good with kappa = 0.79 (0.53–1.00) and 90% accuracy. CONCLUSIONS: Based on these study findings, the WHO cone bioassay is a reproducible bioassay for ITNs with > 80% M24, and for all ITNs provided inherent stochastic variation and systematic bias are accounted for. The literature review confirms that WHO cone bioassay bioefficacy criteria have been previously achieved by all pyrethroid ITNs (unwashed), without the need for additional tunnel tests. The 80% M24 threshold remains the most reliable indicator of pyrethroid ITN quality using pyrethroid susceptible mosquitoes. In the absence of alternative tests, cone bioassays could be used as part of pre-delivery QA. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12936-022-04217-3.