Cargando…

Simulating bout-and-pause patterns with reinforcement learning

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies h...

Descripción completa

Detalles Bibliográficos
Autores principales: Yamada, Kota, Kanemura, Atsunori
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7660465/
https://www.ncbi.nlm.nih.gov/pubmed/33180864
http://dx.doi.org/10.1371/journal.pone.0242201
_version_ 1783609009208557568
author Yamada, Kota
Kanemura, Atsunori
author_facet Yamada, Kota
Kanemura, Atsunori
author_sort Yamada, Kota
collection PubMed
description Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.
format Online
Article
Text
id pubmed-7660465
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-76604652020-11-18 Simulating bout-and-pause patterns with reinforcement learning Yamada, Kota Kanemura, Atsunori PLoS One Research Article Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms. Public Library of Science 2020-11-12 /pmc/articles/PMC7660465/ /pubmed/33180864 http://dx.doi.org/10.1371/journal.pone.0242201 Text en © 2020 Yamada, Kanemura http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Yamada, Kota
Kanemura, Atsunori
Simulating bout-and-pause patterns with reinforcement learning
title Simulating bout-and-pause patterns with reinforcement learning
title_full Simulating bout-and-pause patterns with reinforcement learning
title_fullStr Simulating bout-and-pause patterns with reinforcement learning
title_full_unstemmed Simulating bout-and-pause patterns with reinforcement learning
title_short Simulating bout-and-pause patterns with reinforcement learning
title_sort simulating bout-and-pause patterns with reinforcement learning
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7660465/
https://www.ncbi.nlm.nih.gov/pubmed/33180864
http://dx.doi.org/10.1371/journal.pone.0242201
work_keys_str_mv AT yamadakota simulatingboutandpausepatternswithreinforcementlearning
AT kanemuraatsunori simulatingboutandpausepatternswithreinforcementlearning