Exploring the spatial dimension of estrogen and progesterone signaling: detection of nuclear labeling in lobular epithelial cells in normal mammary glands adjacent to breast cancer

BACKGROUND: Comprehensive spatial assessment of hormone receptor immunohistochemistry staining in digital whole slide images of breast cancer requires accurate detection of positive nuclei within biologically relevant regions of interest. Herein, we propose a combination of automated region labeling at low resolution and subsequent detailed tissue evaluation of subcellular structures in lobular structures adjacent to breast cancer, as a proof of concept for the approach to analyze estrogen and progesterone receptor expression in the spatial context of surrounding tissue. METHODS: Routinely processed paraffin sections of hormone receptor-negative ductal invasive breast cancer were stained for estrogen and progesterone receptor by immunohistochemistry. Digital whole slides were analyzed using commercially available image analysis software for advanced object-based analysis, applying textural, relational, and geometrical features. Mammary gland lobules were targeted as regions of interest for analysis at subcellular level in relation to their distance from coherent tumor as neighboring relevant tissue compartment. Lobule detection quality was evaluated visually by a pathologist. RESULTS: After rule set optimization in an estrogen receptor-stained training set, independent test sets (progesterone and estrogen receptor) showed acceptable detection quality in 33% of cases. Presence of disrupted lobular structures, either by brisk inflammatory infiltrate, or diffuse tumor infiltration, was common in cases with lower detection accuracy. Hormone receptor detection tended towards higher percentage of positively stained nuclei in lobules distant from the tumor border as compared to areas adjacent to the tumor. After adaptations of image analysis, corresponding evaluations were also feasible in hormone receptor positive breast cancer, with some limitations of automated separation of mammary epithelial cells from hormone receptor-positive tumor cells. CONCLUSIONS: As a proof of concept for object-oriented detection of steroid hormone receptors in their spatial context, we show that lobular structures can be classified based on texture-based image features, unless brisk inflammatory infiltration disrupts the normal morphological structure of the tubular gland epithelium. We consider this approach as prototypic for detection and spatial analysis of nuclear markers in defined regions of interest. We conclude that advanced image analysis at this level of complexity requires adaptation to the individual tumor phenotypes and morphological characteristics of the tumor environment.