Cargando…

A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies

Automatic alignment of brain anatomy in a standard space is a key step when processing magnetic resonance imaging for group analyses. Such brain registration is prone to failure, and the results are therefore typically reviewed visually to ensure quality. There is however no standard, validated prot...

Descripción completa

Detalles Bibliográficos
Autores principales: Benhajali, Yassine, Badhwar, AmanPreet, Spiers, Helen, Urchs, Sebastian, Armoza, Jonathan, Ong, Thomas, Pérusse, Daniel, Bellec, Pierre
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7059806/
https://www.ncbi.nlm.nih.gov/pubmed/32180712
http://dx.doi.org/10.3389/fninf.2020.00007
_version_ 1783504123351531520
author Benhajali, Yassine
Badhwar, AmanPreet
Spiers, Helen
Urchs, Sebastian
Armoza, Jonathan
Ong, Thomas
Pérusse, Daniel
Bellec, Pierre
author_facet Benhajali, Yassine
Badhwar, AmanPreet
Spiers, Helen
Urchs, Sebastian
Armoza, Jonathan
Ong, Thomas
Pérusse, Daniel
Bellec, Pierre
author_sort Benhajali, Yassine
collection PubMed
description Automatic alignment of brain anatomy in a standard space is a key step when processing magnetic resonance imaging for group analyses. Such brain registration is prone to failure, and the results are therefore typically reviewed visually to ensure quality. There is however no standard, validated protocol available to perform this visual quality control (QC). We propose here a standardized QC protocol for brain registration, with minimal training overhead and no required knowledge of brain anatomy. We validated the reliability of three-level QC ratings (OK, Maybe, Fail) across different raters. Nine experts each rated N = 100 validation images, and reached moderate to good agreement (kappa from 0.4 to 0.68, average of 0.54 ± 0.08), with the highest agreement for “Fail” images (Dice from 0.67 to 0.93, average of 0.8 ± 0.06). We then recruited volunteers through the Zooniverse crowdsourcing platform, and extracted a consensus panel rating for both the Zooniverse raters (N = 41) and the expert raters. The agreement between expert and Zooniverse panels was high (kappa = 0.76). Overall, our protocol achieved a good reliability when performing a two level assessment (Fail vs. OK/Maybe) by an individual rater, or aggregating multiple three-level ratings (OK, Maybe, Fail) from a panel of experts (3 minimum) or non-experts (15 minimum). Our brain registration QC protocol will help standardize QC practices across laboratories, improve the consistency of reporting of QC in publications, and will open the way for QC assessment of large datasets which could be used to train automated QC systems.
format Online
Article
Text
id pubmed-7059806
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-70598062020-03-16 A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies Benhajali, Yassine Badhwar, AmanPreet Spiers, Helen Urchs, Sebastian Armoza, Jonathan Ong, Thomas Pérusse, Daniel Bellec, Pierre Front Neuroinform Neuroscience Automatic alignment of brain anatomy in a standard space is a key step when processing magnetic resonance imaging for group analyses. Such brain registration is prone to failure, and the results are therefore typically reviewed visually to ensure quality. There is however no standard, validated protocol available to perform this visual quality control (QC). We propose here a standardized QC protocol for brain registration, with minimal training overhead and no required knowledge of brain anatomy. We validated the reliability of three-level QC ratings (OK, Maybe, Fail) across different raters. Nine experts each rated N = 100 validation images, and reached moderate to good agreement (kappa from 0.4 to 0.68, average of 0.54 ± 0.08), with the highest agreement for “Fail” images (Dice from 0.67 to 0.93, average of 0.8 ± 0.06). We then recruited volunteers through the Zooniverse crowdsourcing platform, and extracted a consensus panel rating for both the Zooniverse raters (N = 41) and the expert raters. The agreement between expert and Zooniverse panels was high (kappa = 0.76). Overall, our protocol achieved a good reliability when performing a two level assessment (Fail vs. OK/Maybe) by an individual rater, or aggregating multiple three-level ratings (OK, Maybe, Fail) from a panel of experts (3 minimum) or non-experts (15 minimum). Our brain registration QC protocol will help standardize QC practices across laboratories, improve the consistency of reporting of QC in publications, and will open the way for QC assessment of large datasets which could be used to train automated QC systems. Frontiers Media S.A. 2020-02-28 /pmc/articles/PMC7059806/ /pubmed/32180712 http://dx.doi.org/10.3389/fninf.2020.00007 Text en Copyright © 2020 Benhajali, Badhwar, Spiers, Urchs, Armoza, Ong, Pérusse and Bellec. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Benhajali, Yassine
Badhwar, AmanPreet
Spiers, Helen
Urchs, Sebastian
Armoza, Jonathan
Ong, Thomas
Pérusse, Daniel
Bellec, Pierre
A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies
title A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies
title_full A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies
title_fullStr A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies
title_full_unstemmed A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies
title_short A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies
title_sort standardized protocol for efficient and reliable quality control of brain registration in functional mri studies
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7059806/
https://www.ncbi.nlm.nih.gov/pubmed/32180712
http://dx.doi.org/10.3389/fninf.2020.00007
work_keys_str_mv AT benhajaliyassine astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT badhwaramanpreet astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT spiershelen astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT urchssebastian astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT armozajonathan astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT ongthomas astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT perussedaniel astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT bellecpierre astandardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT benhajaliyassine standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT badhwaramanpreet standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT spiershelen standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT urchssebastian standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT armozajonathan standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT ongthomas standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT perussedaniel standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies
AT bellecpierre standardizedprotocolforefficientandreliablequalitycontrolofbrainregistrationinfunctionalmristudies