Cargando…
Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring
STUDY OBJECTIVES: To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers. METHODS: We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6–12 scorers, to compa...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Oxford University Press
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9905781/ https://www.ncbi.nlm.nih.gov/pubmed/35780449 http://dx.doi.org/10.1093/sleep/zsac154 |
_version_ | 1784883873716371456 |
---|---|
author | Bakker, Jessie P Ross, Marco Cerny, Andreas Vasko, Ray Shaw, Edmund Kuna, Samuel Magalang, Ulysses J Punjabi, Naresh M Anderer, Peter |
author_facet | Bakker, Jessie P Ross, Marco Cerny, Andreas Vasko, Ray Shaw, Edmund Kuna, Samuel Magalang, Ulysses J Punjabi, Naresh M Anderer, Peter |
author_sort | Bakker, Jessie P |
collection | PubMed |
description | STUDY OBJECTIVES: To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers. METHODS: We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6–12 scorers, to compare sleep stage probabilities (hypnodensity; i.e. the probability of each sleep stage being assigned to a given epoch) as the primary output, as well as a single sleep stage per epoch assigned by hierarchical majority rule. RESULTS: The percentage of epochs with 100% agreement across scorers was 46 ± 9%, 38 ± 10% and 32 ± 9% for the datasets with 6, 9, and 12 scorers, respectively. The mean intra-class correlation coefficient between sleep stage probabilities from auto- and manual-scoring was 0.91, representing excellent reliability. Within each dataset, agreement between auto-scoring and consensus manual-scoring was significantly higher than agreement between manual-scoring and consensus manual-scoring (0.78 vs. 0.69; 0.74 vs. 0.67; and 0.75 vs. 0.67; all p < 0.01). CONCLUSIONS: Analysis of scoring performed by multiple scorers reveals that sleep stage ambiguity is the rule rather than the exception. Probabilities of the sleep stages determined by artificial intelligence auto-scoring provide an excellent estimate of this ambiguity. Compared to consensus manual-scoring, sleep staging derived from auto-scoring is for each individual PSG noninferior to manual-scoring meaning that auto-scoring output is ready for interpretation without the need for manual adjustment. |
format | Online Article Text |
id | pubmed-9905781 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Oxford University Press |
record_format | MEDLINE/PubMed |
spelling | pubmed-99057812023-02-09 Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring Bakker, Jessie P Ross, Marco Cerny, Andreas Vasko, Ray Shaw, Edmund Kuna, Samuel Magalang, Ulysses J Punjabi, Naresh M Anderer, Peter Sleep Basic Science of Sleep and Circadian Rhythms STUDY OBJECTIVES: To quantify the amount of sleep stage ambiguity across expert scorers and to validate a new auto-scoring platform against sleep staging performed by multiple scorers. METHODS: We applied a new auto-scoring system to three datasets containing 95 PSGs scored by 6–12 scorers, to compare sleep stage probabilities (hypnodensity; i.e. the probability of each sleep stage being assigned to a given epoch) as the primary output, as well as a single sleep stage per epoch assigned by hierarchical majority rule. RESULTS: The percentage of epochs with 100% agreement across scorers was 46 ± 9%, 38 ± 10% and 32 ± 9% for the datasets with 6, 9, and 12 scorers, respectively. The mean intra-class correlation coefficient between sleep stage probabilities from auto- and manual-scoring was 0.91, representing excellent reliability. Within each dataset, agreement between auto-scoring and consensus manual-scoring was significantly higher than agreement between manual-scoring and consensus manual-scoring (0.78 vs. 0.69; 0.74 vs. 0.67; and 0.75 vs. 0.67; all p < 0.01). CONCLUSIONS: Analysis of scoring performed by multiple scorers reveals that sleep stage ambiguity is the rule rather than the exception. Probabilities of the sleep stages determined by artificial intelligence auto-scoring provide an excellent estimate of this ambiguity. Compared to consensus manual-scoring, sleep staging derived from auto-scoring is for each individual PSG noninferior to manual-scoring meaning that auto-scoring output is ready for interpretation without the need for manual adjustment. Oxford University Press 2022-07-03 /pmc/articles/PMC9905781/ /pubmed/35780449 http://dx.doi.org/10.1093/sleep/zsac154 Text en © Sleep Research Society 2022. Published by Oxford University Press on behalf of the Sleep Research Society. https://creativecommons.org/licenses/by/4.0/This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Basic Science of Sleep and Circadian Rhythms Bakker, Jessie P Ross, Marco Cerny, Andreas Vasko, Ray Shaw, Edmund Kuna, Samuel Magalang, Ulysses J Punjabi, Naresh M Anderer, Peter Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
title | Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
title_full | Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
title_fullStr | Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
title_full_unstemmed | Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
title_short | Scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
title_sort | scoring sleep with artificial intelligence enables quantification of sleep stage ambiguity: hypnodensity based on multiple expert scorers and auto-scoring |
topic | Basic Science of Sleep and Circadian Rhythms |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9905781/ https://www.ncbi.nlm.nih.gov/pubmed/35780449 http://dx.doi.org/10.1093/sleep/zsac154 |
work_keys_str_mv | AT bakkerjessiep scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT rossmarco scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT cernyandreas scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT vaskoray scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT shawedmund scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT kunasamuel scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT magalangulyssesj scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT punjabinareshm scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring AT andererpeter scoringsleepwithartificialintelligenceenablesquantificationofsleepstageambiguityhypnodensitybasedonmultipleexpertscorersandautoscoring |