Cargando…

Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results

Functional magnetic resonance (fMRI) is an invaluable tool in studying cognitive processes in vivo. Many recent studies use functional connectivity (FC), partial correlation connectivity (PC), or fMRI-derived brain networks to predict phenotypes with results that sometimes cannot be replicated. At t...

Descripción completa

Detalles Bibliográficos
Autores principales: Orlichenko, Anton, Qu, Gang, Su, Kuan-Jui, Liu, Anqi, Shen, Hui, Deng, Hong-Wen, Wang, Yu-Ping
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Cornell University 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10418521/
https://www.ncbi.nlm.nih.gov/pubmed/37576121
_version_ 1785088284570943488
author Orlichenko, Anton
Qu, Gang
Su, Kuan-Jui
Liu, Anqi
Shen, Hui
Deng, Hong-Wen
Wang, Yu-Ping
author_facet Orlichenko, Anton
Qu, Gang
Su, Kuan-Jui
Liu, Anqi
Shen, Hui
Deng, Hong-Wen
Wang, Yu-Ping
author_sort Orlichenko, Anton
collection PubMed
description Functional magnetic resonance (fMRI) is an invaluable tool in studying cognitive processes in vivo. Many recent studies use functional connectivity (FC), partial correlation connectivity (PC), or fMRI-derived brain networks to predict phenotypes with results that sometimes cannot be replicated. At the same time, FC can be used to identify the same subject from different scans with great accuracy. In this paper, we show a method by which one can unknowingly inflate classification results from 61% accuracy to 86% accuracy by treating longitudinal or contemporaneous scans of the same subject as independent data points. Using the UK Biobank dataset, we find one can achieve the same level of variance explained with 50 training subjects by exploiting identifiability as with 10,000 training subjects without double-dipping. We replicate this effect in four different datasets: the UK Biobank (UKB), the Philadelphia Neurodevelopmental Cohort (PNC), the Bipolar and Schizophrenia Network for Intermediate Phenotypes (BSNIP), and an OpenNeuro Fibromyalgia dataset (Fibro). The unintentional improvement ranges between 7% and 25% in the four datasets. Additionally, we find that by using dynamic functional connectivity (dFC), one can apply this method even when one is limited to a single scan per subject. One major problem is that features such as ROIs or connectivities that are reported alongside inflated results may confuse future work. This article hopes to shed light on how even minor pipeline anomalies may lead to unexpectedly superb results.
format Online
Article
Text
id pubmed-10418521
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Cornell University
record_format MEDLINE/PubMed
spelling pubmed-104185212023-08-12 Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results Orlichenko, Anton Qu, Gang Su, Kuan-Jui Liu, Anqi Shen, Hui Deng, Hong-Wen Wang, Yu-Ping ArXiv Article Functional magnetic resonance (fMRI) is an invaluable tool in studying cognitive processes in vivo. Many recent studies use functional connectivity (FC), partial correlation connectivity (PC), or fMRI-derived brain networks to predict phenotypes with results that sometimes cannot be replicated. At the same time, FC can be used to identify the same subject from different scans with great accuracy. In this paper, we show a method by which one can unknowingly inflate classification results from 61% accuracy to 86% accuracy by treating longitudinal or contemporaneous scans of the same subject as independent data points. Using the UK Biobank dataset, we find one can achieve the same level of variance explained with 50 training subjects by exploiting identifiability as with 10,000 training subjects without double-dipping. We replicate this effect in four different datasets: the UK Biobank (UKB), the Philadelphia Neurodevelopmental Cohort (PNC), the Bipolar and Schizophrenia Network for Intermediate Phenotypes (BSNIP), and an OpenNeuro Fibromyalgia dataset (Fibro). The unintentional improvement ranges between 7% and 25% in the four datasets. Additionally, we find that by using dynamic functional connectivity (dFC), one can apply this method even when one is limited to a single scan per subject. One major problem is that features such as ROIs or connectivities that are reported alongside inflated results may confuse future work. This article hopes to shed light on how even minor pipeline anomalies may lead to unexpectedly superb results. Cornell University 2023-08-02 /pmc/articles/PMC10418521/ /pubmed/37576121 Text en https://creativecommons.org/licenses/by/4.0/This work is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/) , which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, so long as attribution is given to the creator. The license allows for commercial use.
spellingShingle Article
Orlichenko, Anton
Qu, Gang
Su, Kuan-Jui
Liu, Anqi
Shen, Hui
Deng, Hong-Wen
Wang, Yu-Ping
Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
title Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
title_full Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
title_fullStr Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
title_full_unstemmed Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
title_short Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
title_sort identifiability in functional connectivity may unintentionally inflate prediction results
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10418521/
https://www.ncbi.nlm.nih.gov/pubmed/37576121
work_keys_str_mv AT orlichenkoanton identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults
AT qugang identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults
AT sukuanjui identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults
AT liuanqi identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults
AT shenhui identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults
AT denghongwen identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults
AT wangyuping identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults