Cargando…

Regularization for Unsupervised Learning of Optical Flow

Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convol...

Descripción completa

Detalles Bibliográficos
Autores principales: Long, Libo, Lang, Jochen
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143342/
https://www.ncbi.nlm.nih.gov/pubmed/37112421
http://dx.doi.org/10.3390/s23084080
_version_ 1785033828882972672
author Long, Libo
Lang, Jochen
author_facet Long, Libo
Lang, Jochen
author_sort Long, Libo
collection PubMed
description Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net.
format Online
Article
Text
id pubmed-10143342
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-101433422023-04-29 Regularization for Unsupervised Learning of Optical Flow Long, Libo Lang, Jochen Sensors (Basel) Article Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net. MDPI 2023-04-18 /pmc/articles/PMC10143342/ /pubmed/37112421 http://dx.doi.org/10.3390/s23084080 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Long, Libo
Lang, Jochen
Regularization for Unsupervised Learning of Optical Flow
title Regularization for Unsupervised Learning of Optical Flow
title_full Regularization for Unsupervised Learning of Optical Flow
title_fullStr Regularization for Unsupervised Learning of Optical Flow
title_full_unstemmed Regularization for Unsupervised Learning of Optical Flow
title_short Regularization for Unsupervised Learning of Optical Flow
title_sort regularization for unsupervised learning of optical flow
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10143342/
https://www.ncbi.nlm.nih.gov/pubmed/37112421
http://dx.doi.org/10.3390/s23084080
work_keys_str_mv AT longlibo regularizationforunsupervisedlearningofopticalflow
AT langjochen regularizationforunsupervisedlearningofopticalflow