Cargando…
AUBER: Automated BERT regularization
How can we effectively regularize BERT? Although BERT proves its effectiveness in various NLP tasks, it often overfits when there are only a small number of training instances. A promising direction to regularize BERT is based on pruning its attention heads with a proxy score for head importance. Ho...
| Autores principales: | , , |
|---|---|
| Formato: | Online Artículo Texto |
| Lenguaje: | English |
| Publicado: |
Public Library of Science
2021
|
| Materias: | |
| Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8238198/ https://www.ncbi.nlm.nih.gov/pubmed/34181664 http://dx.doi.org/10.1371/journal.pone.0253241 |