Improving Concrete Crack Segmentation Networks through CutMix Data Synthesis and Temporal Data Fusion

A key element in an automated visual inspection system for concrete structures is identifying the geometric properties of surface defects such as cracks. Fully convolutional neural networks (FCNs) have been demonstrated to be powerful tools for crack segmentation in inspection images. However, the performance of FCNs depends on the size of the dataset that they are trained with. In the absence of large datasets of labeled images for concrete crack segmentation, these networks may lose their excellent prediction accuracy when tested on a new target dataset with different image conditions. In this study, firstly, a Transfer Learning approach is developed to enable the networks better distinguish cracks from background pixels. A synthetic dataset is generated and utilized to fine-tune a U-Net that is pre-trained with a public dataset. In the proposed data synthesis approach, which is based on CutMix data augmentation, the crack images from the public dataset are combined with the background images of a potential target dataset. Secondly, since cracks propagate over time, for sequential images of concrete surfaces, a novel temporal data fusion technique is proposed. In this technique, the network’s predictions from multiple time steps are aggregated to improve the recall of predictions. It is shown that application of the proposed improvements has increased the F1-score and mIoU by 28.4% and 22.2%, respectively, which is a significant enhancement in performance of the segmentation network.