Fine-Grained 3D Modeling and Semantic Mapping of Coral Reefs Using Photogrammetric Computer Vision and Machine Learning †

Corals play a crucial role as the primary habitat-building organisms within reef ecosystems, forming expansive structures that extend over vast distances, akin to the way tall buildings define a city’s skyline. However, coral reefs are vulnerable to damage and destruction due to their inherent fragility and exposure to various threats, including the impacts of climate change. Similar to successful city management, the utilization of advanced underwater videography, photogrammetric computer vision, and machine learning can facilitate precise 3D modeling and the semantic mapping of coral reefs, aiding in their careful management and conservation to ensure their survival. This study focuses on generating detailed 3D mesh models, digital surface models, and orthomosaics of coral habitats by utilizing underwater coral images and control points. Furthermore, an innovative multi-modal deep neural network is designed to perform the pixel-wise semantic segmentation of orthomosaics, enabling the projection of resulting semantic maps onto a 3D space. Notably, this study achieves a significant milestone by accomplishing semantic fine-grained 3D modeling and rugosity evaluation of coral reefs with millimeter-level accuracy, providing a potent means to understand coral reef variations under climate change with high spatial and temporal resolution.