Cargando…

Environment-Aware Rendering and Interaction in Web-Based Augmented Reality

This work presents a novel framework for web-based environment-aware rendering and interaction in augmented reality based on WebXR and three.js. It aims at accelerating the development of device-agnostic Augmented Reality (AR) applications. The solution allows for a realistic rendering of 3D element...

Descripción completa

Detalles Bibliográficos
Autores principales: Ferrão, José, Dias, Paulo, Santos, Beatriz Sousa, Oliveira, Miguel
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10057055/
https://www.ncbi.nlm.nih.gov/pubmed/36976114
http://dx.doi.org/10.3390/jimaging9030063
_version_ 1785016269447102464
author Ferrão, José
Dias, Paulo
Santos, Beatriz Sousa
Oliveira, Miguel
author_facet Ferrão, José
Dias, Paulo
Santos, Beatriz Sousa
Oliveira, Miguel
author_sort Ferrão, José
collection PubMed
description This work presents a novel framework for web-based environment-aware rendering and interaction in augmented reality based on WebXR and three.js. It aims at accelerating the development of device-agnostic Augmented Reality (AR) applications. The solution allows for a realistic rendering of 3D elements, handles geometry occlusion, casts shadows of virtual objects onto real surfaces, and provides physics interaction with real-world objects. Unlike most existing state-of-the-art systems that are built to run on a specific hardware configuration, the proposed solution targets the web environment and is designed to work on a vast range of devices and configurations. Our solution can use monocular camera setups with depth data estimated by deep neural networks or, when available, use higher-quality depth sensors (e.g., LIDAR, structured light) that provide a more accurate perception of the environment. To ensure consistency in the rendering of the virtual scene a physically based rendering pipeline is used, in which physically correct attributes are associated with each 3D object, which, combined with lighting information captured by the device, enables the rendering of AR content matching the environment illumination. All these concepts are integrated and optimized into a pipeline capable of providing a fluid user experience even on middle-range devices. The solution is distributed as an open-source library that can be integrated into existing and new web-based AR projects. The proposed framework was evaluated and compared in terms of performance and visual features with two state-of-the-art alternatives.
format Online
Article
Text
id pubmed-10057055
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-100570552023-03-30 Environment-Aware Rendering and Interaction in Web-Based Augmented Reality Ferrão, José Dias, Paulo Santos, Beatriz Sousa Oliveira, Miguel J Imaging Article This work presents a novel framework for web-based environment-aware rendering and interaction in augmented reality based on WebXR and three.js. It aims at accelerating the development of device-agnostic Augmented Reality (AR) applications. The solution allows for a realistic rendering of 3D elements, handles geometry occlusion, casts shadows of virtual objects onto real surfaces, and provides physics interaction with real-world objects. Unlike most existing state-of-the-art systems that are built to run on a specific hardware configuration, the proposed solution targets the web environment and is designed to work on a vast range of devices and configurations. Our solution can use monocular camera setups with depth data estimated by deep neural networks or, when available, use higher-quality depth sensors (e.g., LIDAR, structured light) that provide a more accurate perception of the environment. To ensure consistency in the rendering of the virtual scene a physically based rendering pipeline is used, in which physically correct attributes are associated with each 3D object, which, combined with lighting information captured by the device, enables the rendering of AR content matching the environment illumination. All these concepts are integrated and optimized into a pipeline capable of providing a fluid user experience even on middle-range devices. The solution is distributed as an open-source library that can be integrated into existing and new web-based AR projects. The proposed framework was evaluated and compared in terms of performance and visual features with two state-of-the-art alternatives. MDPI 2023-03-08 /pmc/articles/PMC10057055/ /pubmed/36976114 http://dx.doi.org/10.3390/jimaging9030063 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Ferrão, José
Dias, Paulo
Santos, Beatriz Sousa
Oliveira, Miguel
Environment-Aware Rendering and Interaction in Web-Based Augmented Reality
title Environment-Aware Rendering and Interaction in Web-Based Augmented Reality
title_full Environment-Aware Rendering and Interaction in Web-Based Augmented Reality
title_fullStr Environment-Aware Rendering and Interaction in Web-Based Augmented Reality
title_full_unstemmed Environment-Aware Rendering and Interaction in Web-Based Augmented Reality
title_short Environment-Aware Rendering and Interaction in Web-Based Augmented Reality
title_sort environment-aware rendering and interaction in web-based augmented reality
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10057055/
https://www.ncbi.nlm.nih.gov/pubmed/36976114
http://dx.doi.org/10.3390/jimaging9030063
work_keys_str_mv AT ferraojose environmentawarerenderingandinteractioninwebbasedaugmentedreality
AT diaspaulo environmentawarerenderingandinteractioninwebbasedaugmentedreality
AT santosbeatrizsousa environmentawarerenderingandinteractioninwebbasedaugmentedreality
AT oliveiramiguel environmentawarerenderingandinteractioninwebbasedaugmentedreality