A novel ground truth dataset enables robust 3D nuclear instance segmentation in early mouse embryos

For investigations into fate specification and cell rearrangements in live images of preimplantation embryos, automated and accurate 3D instance segmentation of nuclei is invaluable; however, the performance of segmentation methods is limited by the images’ low signal-to-noise ratio and high voxel anisotropy and the nuclei’s dense packing and variable shapes. Supervised machine learning approaches have the potential to radically improve segmentation accuracy but are hampered by a lack of fully annotated 3D data. In this work, we first establish a novel mouse line expressing near-infrared nuclear reporter H2B-miRFP720. H2B-miRFP720 is the longest wavelength nuclear reporter in mice and can be imaged simultaneously with other reporters with minimal overlap. We then generate a dataset, which we call BlastoSPIM, of 3D microscopy images of H2B-miRFP720-expressing embryos with ground truth for nuclear instance segmentation. Using BlastoSPIM, we benchmark the performance of five convolutional neural networks and identify Stardist-3D as the most accurate instance segmentation method across preimplantation development. Stardist-3D, trained on BlastoSPIM, performs robustly up to the end of preimplantation development (> 100 nuclei) and enables studies of fate patterning in the late blastocyst. We, then, demonstrate BlastoSPIM’s usefulness as pre-train data for related problems. BlastoSPIM and its corresponding Stardist-3D models are available at: blastospim.flatironinstitute.org.