CT metal artifact reduction algorithms: Toward a framework for objective performance assessment

PURPOSE: Although several metal artifact reduction (MAR) algorithms for computed tomography (CT) scanning are commercially available, no quantitative, rigorous, and reproducible method exists for assessing their performance. The lack of assessment methods poses a challenge to regulators, consumers, and industry. We explored a phantom‐based framework for assessing an important aspect of MAR performance: how applying MAR in the presence of metal affects model observer performance at a low‐contrast detectability (LCD) task This work is, to our knowledge, the first model observer–based framework for the evaluation of MAR algorithms in the published literature. METHODS: We designed a numerical head phantom with metal implants. In order to incorporate an element of randomness, the phantom included a rotatable inset with an inhomogeneous background. We generated simulated projection data for the phantom. We applied two variants of a simple MAR algorithm, sinogram inpainting, to the projection data, that we reconstructed using filtered backprojection. To assess how MAR affected observer performance, we examined the detectability of a signal at the center of a region of interest (ROI) by a channelized Hotelling observer (CHO). As a figure of merit, we used the area under the ROC curve (AUC). RESULTS: We used simulation to test our framework on two variants of the MAR technique of sinogram inpainting. We found that our method was able to resolve the difference in two different MAR algorithms’ effect on LCD task performance, as well as the difference in task performances when MAR was applied, vs not. CONCLUSION: We laid out a phantom‐based framework for objective assessment of how MAR impacts low‐contrast detectability, that we tested on two MAR algorithms. Our results demonstrate the importance of testing MAR performance over a range of object and imaging parameters, since applying MAR does not always improve the quality of an image for a given diagnostic task. Our framework is an initial step toward developing a more comprehensive objective assessment method for MAR, which would require developing additional phantoms and methods specific to various clinical applications of MAR, and increasing study efficiency.