Cell nuclei detection and segmentation for computational pathology using deep learning

<p>Kemeng Chen, Ning Zhang, Janet M Roveda and Linda S Powers</p>

Cell nuclei detection and segmentation for computational pathology using deep learning

Kemeng Chen, Ning Zhang, Janet M Roveda and Linda S Powers

University of Arizona, USA

: J Diagnos Tech Biomed Anal

Abstract

The detection and segmentation of nuclei constitutes an important step in pathology based diagnoses, such as cancer diagnosis, grading and quantitatively analysis, which influence decision reliability. However, manual segmentation is intensive and time consuming work for pathologists. Thus, computer based methods have been introduced to automate this process. However, automated nuclei detection and segmentation is faced with multiple challenges. For example, nuclei exhibit very different morphologies that may depend on the cell type, disease state, cell life cycle, etc., and exist in backgrounds of various tissue morphologies which makes it difficult to detect and segment them. In addition, nuclei often exist close to each other which poses a challenge for their separation. This work implements a deep learning based model incorporating image processing to detect and segment nucleus from microscopic images. The model separates closely spaced nuclei by using a customized loss function which puts more emphasis on adjacent edges. We also incorporate image processing to assist in locating the geometric center of each nuclei and isolate overlapping nuclei. The proposed model has been implemented and tested using an H&E stained dataset containing multiple types of tissues available at. Preliminary results show that the proposed model achieves a recall of 0.955, precision of 0.799, F-Score of 0.866, and mask Intersection over Union (IoU) of 0.83.