A Comprehensive Benchmark of Transcriptomic Biomarkers for Immune Checkpoint Blockades

SIMPLE SUMMARY: Immune checkpoint blockades (ICBs) therapy has produced durable clinical responses in many cancer types, but only a fraction of patients can benefit from ICB treatment. Previous studies have reported multiple transcriptomic biomarkers to predict ICB responses and improve treatment precision in various cancer types. However, a timely and unbiased assessment of these biomarkers has yet to be conducted due to the lack of large-scale uniformly curated ICB-treated datasets. To address the needs, we developed ICB-Portal, a comprehensive resource about ICB including RNA-seq data of 29 datasets from public sources and standardized metadata of each study through a uniform pre-processing, 48 biomarker scores associated with ICB response, results of a systematic benchmark analysis evaluating the efficacy, and generalization ability for each biomarker in various scenarios such as different cancer types, anti-bodies, biopsy time, and combinatory treatments with other drugs by a standardized bioinformatics workflow and an online benchmark platform. ABSTRACT: Immune checkpoint blockades (ICBs) have revolutionized cancer therapy by inducing durable clinical responses, but only a small percentage of patients can benefit from ICB treatments. Many studies have established various biomarkers to predict ICB responses. However, different biomarkers were found with diverse performances in practice, and a timely and unbiased assessment has yet to be conducted due to the complexity of ICB-related studies and trials. In this study, we manually curated 29 published datasets with matched transcriptome and clinical data from more than 1400 patients, and uniformly preprocessed these datasets for further analyses. In addition, we collected 39 sets of transcriptomic biomarkers, and based on the nature of the corresponding computational methods, we categorized them into the gene-set-like group (with the self-contained design and the competitive design, respectively) and the deconvolution-like group. Next, we investigated the correlations and patterns of these biomarkers and utilized a standardized workflow to systematically evaluate their performance in predicting ICB responses and survival statuses across different datasets, cancer types, antibodies, biopsy times, and combinatory treatments. In our benchmark, most biomarkers showed poor performance in terms of stability and robustness across different datasets. Two scores (TIDE and CYT) had a competitive performance for ICB response prediction, and two others (PASS-ON and EIGS_ssGSEA) showed the best association with clinical outcome. Finally, we developed ICB-Portal to host the datasets, biomarkers, and benchmark results and to implement the computational methods for researchers to test their custom biomarkers. Our work provided valuable resources and a one-stop solution to facilitate ICB-related research.