Cargando…
Regularization for Unsupervised Learning of Optical Flow
Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convol...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143342/ https://www.ncbi.nlm.nih.gov/pubmed/37112421 http://dx.doi.org/10.3390/s23084080 |
_version_ | 1785033828882972672 |
---|---|
author | Long, Libo Lang, Jochen |
author_facet | Long, Libo Lang, Jochen |
author_sort | Long, Libo |
collection | PubMed |
description | Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net. |
format | Online Article Text |
id | pubmed-10143342 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-101433422023-04-29 Regularization for Unsupervised Learning of Optical Flow Long, Libo Lang, Jochen Sensors (Basel) Article Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net. MDPI 2023-04-18 /pmc/articles/PMC10143342/ /pubmed/37112421 http://dx.doi.org/10.3390/s23084080 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Long, Libo Lang, Jochen Regularization for Unsupervised Learning of Optical Flow |
title | Regularization for Unsupervised Learning of Optical Flow |
title_full | Regularization for Unsupervised Learning of Optical Flow |
title_fullStr | Regularization for Unsupervised Learning of Optical Flow |
title_full_unstemmed | Regularization for Unsupervised Learning of Optical Flow |
title_short | Regularization for Unsupervised Learning of Optical Flow |
title_sort | regularization for unsupervised learning of optical flow |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143342/ https://www.ncbi.nlm.nih.gov/pubmed/37112421 http://dx.doi.org/10.3390/s23084080 |
work_keys_str_mv | AT longlibo regularizationforunsupervisedlearningofopticalflow AT langjochen regularizationforunsupervisedlearningofopticalflow |