Cargando…

A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade

Radiomics applied in MRI has shown promising results in classifying prostate cancer lesions. However, many papers describe single-center studies without external validation. The issues of using radiomics models on unseen data have not yet been sufficiently addressed. The aim of this study is to eval...

Descripción completa

Detalles Bibliográficos
Autores principales: Castillo T., Jose M., Starmans, Martijn P. A., Arif, Muhammad, Niessen, Wiro J., Klein, Stefan, Bangma, Chris H., Schoots, Ivo G., Veenland, Jifke F.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7926758/
https://www.ncbi.nlm.nih.gov/pubmed/33671533
http://dx.doi.org/10.3390/diagnostics11020369
_version_ 1783659535463874560
author Castillo T., Jose M.
Starmans, Martijn P. A.
Arif, Muhammad
Niessen, Wiro J.
Klein, Stefan
Bangma, Chris H.
Schoots, Ivo G.
Veenland, Jifke F.
author_facet Castillo T., Jose M.
Starmans, Martijn P. A.
Arif, Muhammad
Niessen, Wiro J.
Klein, Stefan
Bangma, Chris H.
Schoots, Ivo G.
Veenland, Jifke F.
author_sort Castillo T., Jose M.
collection PubMed
description Radiomics applied in MRI has shown promising results in classifying prostate cancer lesions. However, many papers describe single-center studies without external validation. The issues of using radiomics models on unseen data have not yet been sufficiently addressed. The aim of this study is to evaluate the generalizability of radiomics models for prostate cancer classification and to compare the performance of these models to the performance of radiologists. Multiparametric MRI, photographs and histology of radical prostatectomy specimens, and pathology reports of 107 patients were obtained from three healthcare centers in the Netherlands. By spatially correlating the MRI with histology, 204 lesions were identified. For each lesion, radiomics features were extracted from the MRI data. Radiomics models for discriminating high-grade (Gleason score ≥ 7) versus low-grade lesions were automatically generated using open-source machine learning software. The performance was tested both in a single-center setting through cross-validation and in a multi-center setting using the two unseen datasets as external validation. For comparison with clinical practice, a multi-center classifier was tested and compared with the Prostate Imaging Reporting and Data System version 2 (PIRADS v2) scoring performed by two expert radiologists. The three single-center models obtained a mean AUC of 0.75, which decreased to 0.54 when the model was applied to the external data, the radiologists obtained a mean AUC of 0.46. In the multi-center setting, the radiomics model obtained a mean AUC of 0.75 while the radiologists obtained a mean AUC of 0.47 on the same subset. While radiomics models have a decent performance when tested on data from the same center(s), they may show a significant drop in performance when applied to external data. On a multi-center dataset our radiomics model outperformed the radiologists, and thus, may represent a more accurate alternative for malignancy prediction.
format Online
Article
Text
id pubmed-7926758
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-79267582021-03-04 A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade Castillo T., Jose M. Starmans, Martijn P. A. Arif, Muhammad Niessen, Wiro J. Klein, Stefan Bangma, Chris H. Schoots, Ivo G. Veenland, Jifke F. Diagnostics (Basel) Article Radiomics applied in MRI has shown promising results in classifying prostate cancer lesions. However, many papers describe single-center studies without external validation. The issues of using radiomics models on unseen data have not yet been sufficiently addressed. The aim of this study is to evaluate the generalizability of radiomics models for prostate cancer classification and to compare the performance of these models to the performance of radiologists. Multiparametric MRI, photographs and histology of radical prostatectomy specimens, and pathology reports of 107 patients were obtained from three healthcare centers in the Netherlands. By spatially correlating the MRI with histology, 204 lesions were identified. For each lesion, radiomics features were extracted from the MRI data. Radiomics models for discriminating high-grade (Gleason score ≥ 7) versus low-grade lesions were automatically generated using open-source machine learning software. The performance was tested both in a single-center setting through cross-validation and in a multi-center setting using the two unseen datasets as external validation. For comparison with clinical practice, a multi-center classifier was tested and compared with the Prostate Imaging Reporting and Data System version 2 (PIRADS v2) scoring performed by two expert radiologists. The three single-center models obtained a mean AUC of 0.75, which decreased to 0.54 when the model was applied to the external data, the radiologists obtained a mean AUC of 0.46. In the multi-center setting, the radiomics model obtained a mean AUC of 0.75 while the radiologists obtained a mean AUC of 0.47 on the same subset. While radiomics models have a decent performance when tested on data from the same center(s), they may show a significant drop in performance when applied to external data. On a multi-center dataset our radiomics model outperformed the radiologists, and thus, may represent a more accurate alternative for malignancy prediction. MDPI 2021-02-22 /pmc/articles/PMC7926758/ /pubmed/33671533 http://dx.doi.org/10.3390/diagnostics11020369 Text en © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Castillo T., Jose M.
Starmans, Martijn P. A.
Arif, Muhammad
Niessen, Wiro J.
Klein, Stefan
Bangma, Chris H.
Schoots, Ivo G.
Veenland, Jifke F.
A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade
title A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade
title_full A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade
title_fullStr A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade
title_full_unstemmed A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade
title_short A Multi-Center, Multi-Vendor Study to Evaluate the Generalizability of a Radiomics Model for Classifying Prostate cancer: High Grade vs. Low Grade
title_sort multi-center, multi-vendor study to evaluate the generalizability of a radiomics model for classifying prostate cancer: high grade vs. low grade
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7926758/
https://www.ncbi.nlm.nih.gov/pubmed/33671533
http://dx.doi.org/10.3390/diagnostics11020369
work_keys_str_mv AT castillotjosem amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT starmansmartijnpa amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT arifmuhammad amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT niessenwiroj amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT kleinstefan amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT bangmachrish amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT schootsivog amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT veenlandjifkef amulticentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT castillotjosem multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT starmansmartijnpa multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT arifmuhammad multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT niessenwiroj multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT kleinstefan multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT bangmachrish multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT schootsivog multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade
AT veenlandjifkef multicentermultivendorstudytoevaluatethegeneralizabilityofaradiomicsmodelforclassifyingprostatecancerhighgradevslowgrade