Cargando…

Automatic landmark detection and mapping for 2D/3D registration with BoneNet

The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) o...

Descripción completa

Detalles Bibliográficos
Autores principales: Nguyen, Van, Alves Pereira, Luis F., Liang, Zhihua, Mielke, Falk, Van Houtte, Jeroen, Sijbers, Jan, De Beenhouwer, Jan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9434378/
https://www.ncbi.nlm.nih.gov/pubmed/36061115
http://dx.doi.org/10.3389/fvets.2022.923449
_version_ 1784780857576259584
author Nguyen, Van
Alves Pereira, Luis F.
Liang, Zhihua
Mielke, Falk
Van Houtte, Jeroen
Sijbers, Jan
De Beenhouwer, Jan
author_facet Nguyen, Van
Alves Pereira, Luis F.
Liang, Zhihua
Mielke, Falk
Van Houtte, Jeroen
Sijbers, Jan
De Beenhouwer, Jan
author_sort Nguyen, Van
collection PubMed
description The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.
format Online
Article
Text
id pubmed-9434378
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-94343782022-09-02 Automatic landmark detection and mapping for 2D/3D registration with BoneNet Nguyen, Van Alves Pereira, Luis F. Liang, Zhihua Mielke, Falk Van Houtte, Jeroen Sijbers, Jan De Beenhouwer, Jan Front Vet Sci Veterinary Science The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations. Frontiers Media S.A. 2022-08-18 /pmc/articles/PMC9434378/ /pubmed/36061115 http://dx.doi.org/10.3389/fvets.2022.923449 Text en Copyright © 2022 Nguyen, Alves Pereira, Liang, Mielke, Van Houtte, Sijbers and De Beenhouwer. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Veterinary Science
Nguyen, Van
Alves Pereira, Luis F.
Liang, Zhihua
Mielke, Falk
Van Houtte, Jeroen
Sijbers, Jan
De Beenhouwer, Jan
Automatic landmark detection and mapping for 2D/3D registration with BoneNet
title Automatic landmark detection and mapping for 2D/3D registration with BoneNet
title_full Automatic landmark detection and mapping for 2D/3D registration with BoneNet
title_fullStr Automatic landmark detection and mapping for 2D/3D registration with BoneNet
title_full_unstemmed Automatic landmark detection and mapping for 2D/3D registration with BoneNet
title_short Automatic landmark detection and mapping for 2D/3D registration with BoneNet
title_sort automatic landmark detection and mapping for 2d/3d registration with bonenet
topic Veterinary Science
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9434378/
https://www.ncbi.nlm.nih.gov/pubmed/36061115
http://dx.doi.org/10.3389/fvets.2022.923449
work_keys_str_mv AT nguyenvan automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet
AT alvespereiraluisf automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet
AT liangzhihua automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet
AT mielkefalk automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet
AT vanhouttejeroen automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet
AT sijbersjan automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet
AT debeenhouwerjan automaticlandmarkdetectionandmappingfor2d3dregistrationwithbonenet