Modeling combined ultrasound and photoacoustic imaging: Simulations aiding device development and artificial intelligence

Combined ultrasound and photoacoustic (USPA) imaging has attracted several pre-clinical and clinical applications due to its ability to simultaneously display structural, functional, and molecular information of deep biological tissue in real time. However, the depth and wavelength dependent optical attenuation and the unknown optical and acoustic heterogeneities limit the USPA imaging performance in deep tissue regions. Novel instrumentation, image reconstruction, and artificial intelligence (AI) methods are currently being investigated to overcome these limitations and improve the USPA image quality. Effective implementation of these approaches requires a reliable USPA simulation tool capable of generating US based anatomical and PA based molecular contrasts of deep biological tissue. Here, we developed a hybrid USPA simulation platform by integrating finite element models of light (NIRFast) and ultrasound (k-Wave) propagations for co-simulation of B-mode US and PA images. The platform allows optimization of different design parameters for USPA devices, such as the aperture size and frequency of both light and ultrasound detector arrays. For designing tissue-realistic digital phantoms, a dictionary-based function has been added to k-Wave to generate various levels of ultrasound speckle contrast. The feasibility of modeling US imaging combined with optical fluence dependent multispectral PA imaging is demonstrated using homogeneous as well as heterogeneous tissue phantoms mimicking human organs (e.g., prostate and finger). In addition, we also demonstrate the potential of the simulation platform to generate large scale application-specific training and test datasets for AI enhanced USPA imaging. The complete USPA simulation codes together with the supplementary user guides have been posted to an open-source repository (https://github.com/KothapalliLabPSU/US-PA_simulation_codes).