Multilaboratory assessment of metagenomic next-generation sequencing for unbiased microbe detection

INTRODUCTION: Metagenomic next-generation sequencing (mNGS) assay for detecting infectious agents is now in the stage of being translated into clinical practice. With no approved approaches or guidelines available, laboratories adopt customized mNGS assays to detect clinical samples. However, the accuracy, reliability, and problems of these routinely implemented assays are not clear. OBJECTIVES: To evaluate the performance of 90 mNGS laboratories under routine testing conditions through analyzing identical samples. METHODS: Eleven microbial communities were generated using 15 quantitative microbial suspensions. They were used as reference materials to evaluate the false negatives and false positives of participating mNGS protocols, as well as the ability to distinguish genetically similar organisms and to identify true pathogens from other microbes based on fictitious case reports. RESULTS: High interlaboratory variability was found in the identification and the quantitative reads per million reads (RPM) values of each microbe in the samples, especially when testing microbes present at low concentrations (1 × 10(3) cell/ml or less). 42.2% (38/90) of the laboratories reported unexpected microbes (i.e. false positive problem). Only 56.7% (51/90) to 83.3% (75/90) of the laboratories showed a sufficient ability to obtain clear etiological diagnoses for three simulated cases combined with patient information. The analysis of the performance of mNGS in distinguishing genetically similar organisms in three samples revealed that only 56.6% to 63.0% of the laboratories recovered RPM ratios (RPM(S. aureus)/RPM(S. epidermidis)) within the range of a 2-fold change of the initial input ratios (indicating a relatively low level of bias). CONCLUSION: The high interlaboratory variability found in both identifying microbes and distinguishing true pathogens emphasizes the urgent need for improving the accuracy and comparability of the results generated across different mNGS laboratories, especially in the detection of low-microbial-biomass samples.