Explainability-Guided Two-Stage Nuclei Segmentation and Classification in Histopathology

Abstract

Deep learning has advanced computational pathology by automating nuclei-level analysis on hematoxylin, eosin slides. This thesis proposes an explainability-guided, two-stage framework that jointly tackles segmentation and phenotype classification while maintaining clinical inter pretability and practical efficiency. In Stage 1, an Attention U-Net with a ResNet-34 encoder is trained on the multi-organ PanNuke dataset using a hybrid loss (Dice 0.7 / Focal 0.2 / Boundary 0.1) with MixUp, EMA, and progressive resizing. On the evaluation split, we obtain mean Dice 0.742 (including background) and 0.698 (excluding background), with mean IoU 0.610 and 0.545, respectively, and Boundary-F1 0.71. In Stage 2, cropped nucleus instances are classified by an EfficientNet-B3 model, yielding overall accuracy 0.834 and macro-F1 0.81 across neoplastic, inflammatory, connective, dead, and epithelial nuclei, with diagonals >75% in the confusion matrix. Interpretability is built-in rather than post-hoc: Grad-CAM and SHAP consistently localize attributions within annotated nuclear boundaries; a localization score of ≈0.82 indicates that most salient evidence lies inside nuclei. The complete pipeline runs in ∼100ms per 256 × 256 patch using ∼1.8GB GPU memory, supporting interactive slide-level exploration. The modular, open implementation and GUI facilitate reproducibility, ablation, and clinical auditing.