Cargando…

Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform

Ultrasonic vocalizations (USVs) are known to reflect emotional processing, brain neurochemistry, and brain function. Collecting and processing USV data is manual, time-intensive, and costly, creating a significant bottleneck by limiting researchers’ ability to employ fully effective and nuanced expe...

Descripción completa

Detalles Bibliográficos
Autores principales: Ashley, Catherine B., Snyder, Ryan D., Shepherd, James E., Cervantes, Catalina, Mittal, Nitish, Fleming, Sheila, Bailey, Jaxon, Nievera, Maisie D., Souleimanova, Sharmin Islam, Nyaoga, Bill, Lichtenfeld, Lauren, Chen, Alicia R., Maddox, W. Todd, Duvauchelle, Christine L.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8301917/
https://www.ncbi.nlm.nih.gov/pubmed/34209754
http://dx.doi.org/10.3390/brainsci11070864
_version_ 1783726782616174592
author Ashley, Catherine B.
Snyder, Ryan D.
Shepherd, James E.
Cervantes, Catalina
Mittal, Nitish
Fleming, Sheila
Bailey, Jaxon
Nievera, Maisie D.
Souleimanova, Sharmin Islam
Nyaoga, Bill
Lichtenfeld, Lauren
Chen, Alicia R.
Maddox, W. Todd
Duvauchelle, Christine L.
author_facet Ashley, Catherine B.
Snyder, Ryan D.
Shepherd, James E.
Cervantes, Catalina
Mittal, Nitish
Fleming, Sheila
Bailey, Jaxon
Nievera, Maisie D.
Souleimanova, Sharmin Islam
Nyaoga, Bill
Lichtenfeld, Lauren
Chen, Alicia R.
Maddox, W. Todd
Duvauchelle, Christine L.
author_sort Ashley, Catherine B.
collection PubMed
description Ultrasonic vocalizations (USVs) are known to reflect emotional processing, brain neurochemistry, and brain function. Collecting and processing USV data is manual, time-intensive, and costly, creating a significant bottleneck by limiting researchers’ ability to employ fully effective and nuanced experimental designs and serving as a barrier to entry for other researchers. In this report, we provide a snapshot of the current development and testing of Acoustilytix™, a web-based automated USV scoring tool. Acoustilytix implements machine learning methodology in the USV detection and classification process and is recording-environment-agnostic. We summarize the user features identified as desirable by USV researchers and how these were implemented. These include the ability to easily upload USV files, output a list of detected USVs with associated parameters in csv format, and the ability to manually verify or modify an automatically detected call. With no user intervention or tuning, Acoustilytix achieves 93% sensitivity (a measure of how accurately Acoustilytix detects true calls) and 73% precision (a measure of how accurately Acoustilytix avoids false positives) in call detection across four unique recording environments and was superior to the popular DeepSqueak algorithm (sensitivity = 88%; precision = 41%). Future work will include integration and implementation of machine-learning-based call type classification prediction that will recommend a call type to the user for each detected call. Call classification accuracy is currently in the 71–79% accuracy range, which will continue to improve as more USV files are scored by expert scorers, providing more training data for the classification model. We also describe a recently developed feature of Acoustilytix that offers a fast and effective way to train hand-scorers using automated learning principles without the need for an expert hand-scorer to be present and is built upon a foundation of learning science. The key is that trainees are given practice classifying hundreds of calls with immediate corrective feedback based on an expert’s USV classification. We showed that this approach is highly effective with inter-rater reliability (i.e., kappa statistics) between trainees and the expert ranging from 0.30–0.75 (average = 0.55) after only 1000–2000 calls of training. We conclude with a brief discussion of future improvements to the Acoustilytix platform.
format Online
Article
Text
id pubmed-8301917
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-83019172021-07-24 Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform Ashley, Catherine B. Snyder, Ryan D. Shepherd, James E. Cervantes, Catalina Mittal, Nitish Fleming, Sheila Bailey, Jaxon Nievera, Maisie D. Souleimanova, Sharmin Islam Nyaoga, Bill Lichtenfeld, Lauren Chen, Alicia R. Maddox, W. Todd Duvauchelle, Christine L. Brain Sci Article Ultrasonic vocalizations (USVs) are known to reflect emotional processing, brain neurochemistry, and brain function. Collecting and processing USV data is manual, time-intensive, and costly, creating a significant bottleneck by limiting researchers’ ability to employ fully effective and nuanced experimental designs and serving as a barrier to entry for other researchers. In this report, we provide a snapshot of the current development and testing of Acoustilytix™, a web-based automated USV scoring tool. Acoustilytix implements machine learning methodology in the USV detection and classification process and is recording-environment-agnostic. We summarize the user features identified as desirable by USV researchers and how these were implemented. These include the ability to easily upload USV files, output a list of detected USVs with associated parameters in csv format, and the ability to manually verify or modify an automatically detected call. With no user intervention or tuning, Acoustilytix achieves 93% sensitivity (a measure of how accurately Acoustilytix detects true calls) and 73% precision (a measure of how accurately Acoustilytix avoids false positives) in call detection across four unique recording environments and was superior to the popular DeepSqueak algorithm (sensitivity = 88%; precision = 41%). Future work will include integration and implementation of machine-learning-based call type classification prediction that will recommend a call type to the user for each detected call. Call classification accuracy is currently in the 71–79% accuracy range, which will continue to improve as more USV files are scored by expert scorers, providing more training data for the classification model. We also describe a recently developed feature of Acoustilytix that offers a fast and effective way to train hand-scorers using automated learning principles without the need for an expert hand-scorer to be present and is built upon a foundation of learning science. The key is that trainees are given practice classifying hundreds of calls with immediate corrective feedback based on an expert’s USV classification. We showed that this approach is highly effective with inter-rater reliability (i.e., kappa statistics) between trainees and the expert ranging from 0.30–0.75 (average = 0.55) after only 1000–2000 calls of training. We conclude with a brief discussion of future improvements to the Acoustilytix platform. MDPI 2021-06-29 /pmc/articles/PMC8301917/ /pubmed/34209754 http://dx.doi.org/10.3390/brainsci11070864 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Ashley, Catherine B.
Snyder, Ryan D.
Shepherd, James E.
Cervantes, Catalina
Mittal, Nitish
Fleming, Sheila
Bailey, Jaxon
Nievera, Maisie D.
Souleimanova, Sharmin Islam
Nyaoga, Bill
Lichtenfeld, Lauren
Chen, Alicia R.
Maddox, W. Todd
Duvauchelle, Christine L.
Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
title Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
title_full Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
title_fullStr Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
title_full_unstemmed Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
title_short Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
title_sort acoustilytix™: a web-based automated ultrasonic vocalization scoring platform
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8301917/
https://www.ncbi.nlm.nih.gov/pubmed/34209754
http://dx.doi.org/10.3390/brainsci11070864
work_keys_str_mv AT ashleycatherineb acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT snyderryand acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT shepherdjamese acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT cervantescatalina acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT mittalnitish acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT flemingsheila acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT baileyjaxon acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT nieveramaisied acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT souleimanovasharminislam acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT nyaogabill acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT lichtenfeldlauren acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT chenaliciar acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT maddoxwtodd acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform
AT duvauchellechristinel acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform