Cargando…
A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning
Humans do not learn everything from the scratch but can connect and associate the upcoming information with the exchanged experience and known knowledge. Such an idea can be extended to cooperated multi-reinforcement learning and has achieved its success on homogeneous agents by means of parameter s...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10315621/ https://www.ncbi.nlm.nih.gov/pubmed/37404464 http://dx.doi.org/10.3389/fnins.2023.1201370 |
_version_ | 1785067536175333376 |
---|---|
author | Zhao, Biao Jin, Weiqiang Chen, Zhang Guo, Yucheng |
author_facet | Zhao, Biao Jin, Weiqiang Chen, Zhang Guo, Yucheng |
author_sort | Zhao, Biao |
collection | PubMed |
description | Humans do not learn everything from the scratch but can connect and associate the upcoming information with the exchanged experience and known knowledge. Such an idea can be extended to cooperated multi-reinforcement learning and has achieved its success on homogeneous agents by means of parameter sharing. However, it is difficult to straightforwardly apply parameter sharing when dealing with heterogeneous agents thanks to their individual forms of input/output and their diverse functions and targets. Neuroscience has provided evidence that our brain creates several levels of experience and knowledge-sharing mechanisms that not only exchange similar experiences but also allow for sharing of abstract concepts to handle unfamiliar situations that others have already encountered. Inspired by such a brain's functions, we propose a semi-independent training policy method that can well tackle the conflict between parameter sharing and specialized training for heterogeneous agents. It employs a shared common representation for both observation and action, enabling the integration of various input and output sources. Additionally, a shared latent space is utilized to maintain a balanced relationship between the upstream policy and downstream functions, benefiting each individual agent's target. From the experiments, it can approve that our proposed method outperforms the current mainstream algorithms, especially when handling heterogeneous agents. Empirically, our proposed method can also be improved as a more general and fundamental heterogeneous agents' reinforcement learning structure for curriculum learning and representation transfer. All our code is open and released on https://gitlab.com/reinforcement/ntype. |
format | Online Article Text |
id | pubmed-10315621 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-103156212023-07-04 A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning Zhao, Biao Jin, Weiqiang Chen, Zhang Guo, Yucheng Front Neurosci Neuroscience Humans do not learn everything from the scratch but can connect and associate the upcoming information with the exchanged experience and known knowledge. Such an idea can be extended to cooperated multi-reinforcement learning and has achieved its success on homogeneous agents by means of parameter sharing. However, it is difficult to straightforwardly apply parameter sharing when dealing with heterogeneous agents thanks to their individual forms of input/output and their diverse functions and targets. Neuroscience has provided evidence that our brain creates several levels of experience and knowledge-sharing mechanisms that not only exchange similar experiences but also allow for sharing of abstract concepts to handle unfamiliar situations that others have already encountered. Inspired by such a brain's functions, we propose a semi-independent training policy method that can well tackle the conflict between parameter sharing and specialized training for heterogeneous agents. It employs a shared common representation for both observation and action, enabling the integration of various input and output sources. Additionally, a shared latent space is utilized to maintain a balanced relationship between the upstream policy and downstream functions, benefiting each individual agent's target. From the experiments, it can approve that our proposed method outperforms the current mainstream algorithms, especially when handling heterogeneous agents. Empirically, our proposed method can also be improved as a more general and fundamental heterogeneous agents' reinforcement learning structure for curriculum learning and representation transfer. All our code is open and released on https://gitlab.com/reinforcement/ntype. Frontiers Media S.A. 2023-06-19 /pmc/articles/PMC10315621/ /pubmed/37404464 http://dx.doi.org/10.3389/fnins.2023.1201370 Text en Copyright © 2023 Zhao, Jin, Chen and Guo. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Zhao, Biao Jin, Weiqiang Chen, Zhang Guo, Yucheng A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
title | A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
title_full | A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
title_fullStr | A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
title_full_unstemmed | A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
title_short | A semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
title_sort | semi-independent policies training method with shared representation for heterogeneous multi-agents reinforcement learning |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10315621/ https://www.ncbi.nlm.nih.gov/pubmed/37404464 http://dx.doi.org/10.3389/fnins.2023.1201370 |
work_keys_str_mv | AT zhaobiao asemiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT jinweiqiang asemiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT chenzhang asemiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT guoyucheng asemiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT zhaobiao semiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT jinweiqiang semiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT chenzhang semiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning AT guoyucheng semiindependentpoliciestrainingmethodwithsharedrepresentationforheterogeneousmultiagentsreinforcementlearning |