Imitation learning for improved 3D PET/MR attenuation correction()

The assessment of the quality of synthesised/pseudo Computed Tomography (pCT) images is commonly measured by an intensity-wise similarity between the ground truth CT and the pCT. However, when using the pCT as an attenuation map ([Formula: see text]-map) for PET reconstruction in Positron Emission Tomography Magnetic Resonance Imaging (PET/MRI) minimising the error between pCT and CT neglects the main objective of predicting a pCT that when used as [Formula: see text]-map reconstructs a pseudo PET (pPET) which is as similar as possible to the gold standard CT-derived PET reconstruction. This observation motivated us to propose a novel multi-hypothesis deep learning framework explicitly aimed at PET reconstruction application. A convolutional neural network (CNN) synthesises pCTs by minimising a combination of the pixel-wise error between pCT and CT and a novel metric-loss that itself is defined by a CNN and aims to minimise consequent PET residuals. Training is performed on a database of twenty 3D MR/CT/PET brain image pairs. Quantitative results on a fully independent dataset of twenty-three 3D MR/CT/PET image pairs show that the network is able to synthesise more accurate pCTs. The Mean Absolute Error on the pCT (110.98 HU [Formula: see text] 19.22 HU) compared to a baseline CNN (172.12 HU [Formula: see text] 19.61 HU) and a multi-atlas propagation approach (153.40 HU [Formula: see text] 18.68 HU), and subsequently lead to a significant improvement in the PET reconstruction error (4.74% [Formula: see text] 1.52% compared to baseline 13.72% [Formula: see text] 2.48% and multi-atlas propagation 6.68% [Formula: see text] 2.06%).