Cargando…
Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform
Ultrasonic vocalizations (USVs) are known to reflect emotional processing, brain neurochemistry, and brain function. Collecting and processing USV data is manual, time-intensive, and costly, creating a significant bottleneck by limiting researchers’ ability to employ fully effective and nuanced expe...
Autores principales: | , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8301917/ https://www.ncbi.nlm.nih.gov/pubmed/34209754 http://dx.doi.org/10.3390/brainsci11070864 |
_version_ | 1783726782616174592 |
---|---|
author | Ashley, Catherine B. Snyder, Ryan D. Shepherd, James E. Cervantes, Catalina Mittal, Nitish Fleming, Sheila Bailey, Jaxon Nievera, Maisie D. Souleimanova, Sharmin Islam Nyaoga, Bill Lichtenfeld, Lauren Chen, Alicia R. Maddox, W. Todd Duvauchelle, Christine L. |
author_facet | Ashley, Catherine B. Snyder, Ryan D. Shepherd, James E. Cervantes, Catalina Mittal, Nitish Fleming, Sheila Bailey, Jaxon Nievera, Maisie D. Souleimanova, Sharmin Islam Nyaoga, Bill Lichtenfeld, Lauren Chen, Alicia R. Maddox, W. Todd Duvauchelle, Christine L. |
author_sort | Ashley, Catherine B. |
collection | PubMed |
description | Ultrasonic vocalizations (USVs) are known to reflect emotional processing, brain neurochemistry, and brain function. Collecting and processing USV data is manual, time-intensive, and costly, creating a significant bottleneck by limiting researchers’ ability to employ fully effective and nuanced experimental designs and serving as a barrier to entry for other researchers. In this report, we provide a snapshot of the current development and testing of Acoustilytix™, a web-based automated USV scoring tool. Acoustilytix implements machine learning methodology in the USV detection and classification process and is recording-environment-agnostic. We summarize the user features identified as desirable by USV researchers and how these were implemented. These include the ability to easily upload USV files, output a list of detected USVs with associated parameters in csv format, and the ability to manually verify or modify an automatically detected call. With no user intervention or tuning, Acoustilytix achieves 93% sensitivity (a measure of how accurately Acoustilytix detects true calls) and 73% precision (a measure of how accurately Acoustilytix avoids false positives) in call detection across four unique recording environments and was superior to the popular DeepSqueak algorithm (sensitivity = 88%; precision = 41%). Future work will include integration and implementation of machine-learning-based call type classification prediction that will recommend a call type to the user for each detected call. Call classification accuracy is currently in the 71–79% accuracy range, which will continue to improve as more USV files are scored by expert scorers, providing more training data for the classification model. We also describe a recently developed feature of Acoustilytix that offers a fast and effective way to train hand-scorers using automated learning principles without the need for an expert hand-scorer to be present and is built upon a foundation of learning science. The key is that trainees are given practice classifying hundreds of calls with immediate corrective feedback based on an expert’s USV classification. We showed that this approach is highly effective with inter-rater reliability (i.e., kappa statistics) between trainees and the expert ranging from 0.30–0.75 (average = 0.55) after only 1000–2000 calls of training. We conclude with a brief discussion of future improvements to the Acoustilytix platform. |
format | Online Article Text |
id | pubmed-8301917 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-83019172021-07-24 Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform Ashley, Catherine B. Snyder, Ryan D. Shepherd, James E. Cervantes, Catalina Mittal, Nitish Fleming, Sheila Bailey, Jaxon Nievera, Maisie D. Souleimanova, Sharmin Islam Nyaoga, Bill Lichtenfeld, Lauren Chen, Alicia R. Maddox, W. Todd Duvauchelle, Christine L. Brain Sci Article Ultrasonic vocalizations (USVs) are known to reflect emotional processing, brain neurochemistry, and brain function. Collecting and processing USV data is manual, time-intensive, and costly, creating a significant bottleneck by limiting researchers’ ability to employ fully effective and nuanced experimental designs and serving as a barrier to entry for other researchers. In this report, we provide a snapshot of the current development and testing of Acoustilytix™, a web-based automated USV scoring tool. Acoustilytix implements machine learning methodology in the USV detection and classification process and is recording-environment-agnostic. We summarize the user features identified as desirable by USV researchers and how these were implemented. These include the ability to easily upload USV files, output a list of detected USVs with associated parameters in csv format, and the ability to manually verify or modify an automatically detected call. With no user intervention or tuning, Acoustilytix achieves 93% sensitivity (a measure of how accurately Acoustilytix detects true calls) and 73% precision (a measure of how accurately Acoustilytix avoids false positives) in call detection across four unique recording environments and was superior to the popular DeepSqueak algorithm (sensitivity = 88%; precision = 41%). Future work will include integration and implementation of machine-learning-based call type classification prediction that will recommend a call type to the user for each detected call. Call classification accuracy is currently in the 71–79% accuracy range, which will continue to improve as more USV files are scored by expert scorers, providing more training data for the classification model. We also describe a recently developed feature of Acoustilytix that offers a fast and effective way to train hand-scorers using automated learning principles without the need for an expert hand-scorer to be present and is built upon a foundation of learning science. The key is that trainees are given practice classifying hundreds of calls with immediate corrective feedback based on an expert’s USV classification. We showed that this approach is highly effective with inter-rater reliability (i.e., kappa statistics) between trainees and the expert ranging from 0.30–0.75 (average = 0.55) after only 1000–2000 calls of training. We conclude with a brief discussion of future improvements to the Acoustilytix platform. MDPI 2021-06-29 /pmc/articles/PMC8301917/ /pubmed/34209754 http://dx.doi.org/10.3390/brainsci11070864 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Ashley, Catherine B. Snyder, Ryan D. Shepherd, James E. Cervantes, Catalina Mittal, Nitish Fleming, Sheila Bailey, Jaxon Nievera, Maisie D. Souleimanova, Sharmin Islam Nyaoga, Bill Lichtenfeld, Lauren Chen, Alicia R. Maddox, W. Todd Duvauchelle, Christine L. Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform |
title | Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform |
title_full | Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform |
title_fullStr | Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform |
title_full_unstemmed | Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform |
title_short | Acoustilytix™: A Web-Based Automated Ultrasonic Vocalization Scoring Platform |
title_sort | acoustilytix™: a web-based automated ultrasonic vocalization scoring platform |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8301917/ https://www.ncbi.nlm.nih.gov/pubmed/34209754 http://dx.doi.org/10.3390/brainsci11070864 |
work_keys_str_mv | AT ashleycatherineb acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT snyderryand acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT shepherdjamese acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT cervantescatalina acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT mittalnitish acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT flemingsheila acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT baileyjaxon acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT nieveramaisied acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT souleimanovasharminislam acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT nyaogabill acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT lichtenfeldlauren acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT chenaliciar acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT maddoxwtodd acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform AT duvauchellechristinel acoustilytixawebbasedautomatedultrasonicvocalizationscoringplatform |