Cargando…
Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results
Functional magnetic resonance (fMRI) is an invaluable tool in studying cognitive processes in vivo. Many recent studies use functional connectivity (FC), partial correlation connectivity (PC), or fMRI-derived brain networks to predict phenotypes with results that sometimes cannot be replicated. At t...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Cornell University
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10418521/ https://www.ncbi.nlm.nih.gov/pubmed/37576121 |
_version_ | 1785088284570943488 |
---|---|
author | Orlichenko, Anton Qu, Gang Su, Kuan-Jui Liu, Anqi Shen, Hui Deng, Hong-Wen Wang, Yu-Ping |
author_facet | Orlichenko, Anton Qu, Gang Su, Kuan-Jui Liu, Anqi Shen, Hui Deng, Hong-Wen Wang, Yu-Ping |
author_sort | Orlichenko, Anton |
collection | PubMed |
description | Functional magnetic resonance (fMRI) is an invaluable tool in studying cognitive processes in vivo. Many recent studies use functional connectivity (FC), partial correlation connectivity (PC), or fMRI-derived brain networks to predict phenotypes with results that sometimes cannot be replicated. At the same time, FC can be used to identify the same subject from different scans with great accuracy. In this paper, we show a method by which one can unknowingly inflate classification results from 61% accuracy to 86% accuracy by treating longitudinal or contemporaneous scans of the same subject as independent data points. Using the UK Biobank dataset, we find one can achieve the same level of variance explained with 50 training subjects by exploiting identifiability as with 10,000 training subjects without double-dipping. We replicate this effect in four different datasets: the UK Biobank (UKB), the Philadelphia Neurodevelopmental Cohort (PNC), the Bipolar and Schizophrenia Network for Intermediate Phenotypes (BSNIP), and an OpenNeuro Fibromyalgia dataset (Fibro). The unintentional improvement ranges between 7% and 25% in the four datasets. Additionally, we find that by using dynamic functional connectivity (dFC), one can apply this method even when one is limited to a single scan per subject. One major problem is that features such as ROIs or connectivities that are reported alongside inflated results may confuse future work. This article hopes to shed light on how even minor pipeline anomalies may lead to unexpectedly superb results. |
format | Online Article Text |
id | pubmed-10418521 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Cornell University |
record_format | MEDLINE/PubMed |
spelling | pubmed-104185212023-08-12 Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results Orlichenko, Anton Qu, Gang Su, Kuan-Jui Liu, Anqi Shen, Hui Deng, Hong-Wen Wang, Yu-Ping ArXiv Article Functional magnetic resonance (fMRI) is an invaluable tool in studying cognitive processes in vivo. Many recent studies use functional connectivity (FC), partial correlation connectivity (PC), or fMRI-derived brain networks to predict phenotypes with results that sometimes cannot be replicated. At the same time, FC can be used to identify the same subject from different scans with great accuracy. In this paper, we show a method by which one can unknowingly inflate classification results from 61% accuracy to 86% accuracy by treating longitudinal or contemporaneous scans of the same subject as independent data points. Using the UK Biobank dataset, we find one can achieve the same level of variance explained with 50 training subjects by exploiting identifiability as with 10,000 training subjects without double-dipping. We replicate this effect in four different datasets: the UK Biobank (UKB), the Philadelphia Neurodevelopmental Cohort (PNC), the Bipolar and Schizophrenia Network for Intermediate Phenotypes (BSNIP), and an OpenNeuro Fibromyalgia dataset (Fibro). The unintentional improvement ranges between 7% and 25% in the four datasets. Additionally, we find that by using dynamic functional connectivity (dFC), one can apply this method even when one is limited to a single scan per subject. One major problem is that features such as ROIs or connectivities that are reported alongside inflated results may confuse future work. This article hopes to shed light on how even minor pipeline anomalies may lead to unexpectedly superb results. Cornell University 2023-08-02 /pmc/articles/PMC10418521/ /pubmed/37576121 Text en https://creativecommons.org/licenses/by/4.0/This work is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/) , which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, so long as attribution is given to the creator. The license allows for commercial use. |
spellingShingle | Article Orlichenko, Anton Qu, Gang Su, Kuan-Jui Liu, Anqi Shen, Hui Deng, Hong-Wen Wang, Yu-Ping Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results |
title | Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results |
title_full | Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results |
title_fullStr | Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results |
title_full_unstemmed | Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results |
title_short | Identifiability in Functional Connectivity May Unintentionally Inflate Prediction Results |
title_sort | identifiability in functional connectivity may unintentionally inflate prediction results |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10418521/ https://www.ncbi.nlm.nih.gov/pubmed/37576121 |
work_keys_str_mv | AT orlichenkoanton identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults AT qugang identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults AT sukuanjui identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults AT liuanqi identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults AT shenhui identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults AT denghongwen identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults AT wangyuping identifiabilityinfunctionalconnectivitymayunintentionallyinflatepredictionresults |