Paper deep dive
Hierarchical Multi-Scale Graph Learning with Knowledge-Guided Attention for Whole-Slide Image Survival Analysis
Bin Xu, Yufei Zhou, Boling Song, Jingwen Sun, Yang Bian, Cheng Lu, Ye Wu, Jianfei Tu, Xiangxue Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/20/2026, 8:21:17 AM
Summary
The paper introduces HMKGN, a Hierarchical Multi-Scale Knowledge-aware Graph Network for whole-slide image (WSI) survival analysis. It addresses limitations in existing Multiple Instance Learning (MIL) methods by enforcing spatial locality constraints and modeling multi-scale interactions. HMKGN uses local dynamic graphs for cellular-level patch aggregation within Regions of Interest (ROIs) and global dynamic graphs for slide-level integration. It fuses coarse contextual features from low magnification with fine-grained features from high magnification using bidirectional cross-attention. Evaluated on four TCGA cohorts (KIRC, LGG, PAAD, STAD), HMKGN outperforms baseline MIL and GNN models in concordance index and survival risk stratification.
Entities (14)
Relation Signals (8)
HMKGN → evaluatedon → TCGA
confidence 95% · We evaluate HMKGN on four TCGA cohorts (KIRC, LGG, PAAD, and STAD)
HMKGN → uses → Multi-Scale Feature Fusion
confidence 95% · multi-scale integration at the ROI level combines coarse contextual features from broader views with fine-grained structural representations
HMKGN → uses → Spatial Locality Constraints
confidence 95% · HMKGN enforces a hierarchical structure with spatial locality constraints, wherein local cellular-level dynamic graphs aggregate spatially proximate patches
HMKGN → improves → Concordance Index
confidence 90% · yielding improved concordance indices (10.85% better)
HMKGN → outperforms → WiKG
confidence 90% · HMKGN consistently outperforms existing methods... Table 1... HMKGN ... 0.7486 ... WiKG ... 0.7176
HMKGN → outperforms → TransMIL
confidence 90% · It consistently outperforms existing MIL-based models... Table 1... HMKGN ... 0.7486 ... TransMIL ... 0.5241
Bidirectional Cross-Attention → usedin → HMKGN
confidence 90% · coarse contextual representations ... were fused ... through a bidirectional cross-attention mechanism (BiX)
UNIv2 → usedby → HMKGN
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We propose a Hierarchical Multi-scale Knowledge-aware Graph Network (HMKGN) that models multi-scale interactions and spatially hierarchical relationships within whole-slide images (WSIs) for cancer prognostication. Unlike conventional attention-based MIL, which ignores spatial organization, or graph-based MIL, which relies on static handcrafted graphs, HMKGN enforces a hierarchical structure with spatial locality constraints, wherein local cellular-level dynamic graphs aggregate spatially proximate patches within each region of interest (ROI) and a global slide-level dynamic graph integrates ROI-level features into WSI-level representations. Moreover, multi-scale integration at the ROI level combines coarse contextual features from broader views with fine-grained structural representations from local patch-graph aggregation. We evaluate HMKGN on four TCGA cohorts (KIRC, LGG, PAAD, and STAD; N=513, 487, 138, and 370) for survival prediction. It consistently outperforms existing MIL-based models, yielding improved concordance indices (10.85% better) and statistically significant stratification of patient survival risk (log-rank p < 0.05).
Tags
Links
- Source: https://arxiv.org/abs/2602.23557v2
- Canonical: https://arxiv.org/abs/2602.23557v2
Trouble viewing inline? Open PDF directly →
Full Text
16,883 characters extracted from source content.
Expand or collapse full text
HIERARCHICAL MULTI-SCALE GRAPH LEARNING WITH KNOWLEDGE-GUIDED ATTENTION FOR WHOLE-SLIDE IMAGE SURVIVAL ANALYSIS Bin Xu 1# , Yufei Zhou 2# , Bolin Song 3 , Jingwen Sun 4 , Yang Bian 1 ,Yikai Chen 1 , Cheng Lu 5 , Ye Wu 6 , Jianfei Tu 7 , Xiangxue Wang 1* 1. N anjing University of Information Science and Technology, Nanjing, China 2. C ase Western Reserve University, Cleveland, OH, USA 3. G eorgia Institute of Technology and Emory University, Atlanta, GA, USA 4. Nanjing Women and Children's Healthcare Hospital, Nanjing, China 5. G uangdong Provincial People's Hospital, Guangzhou, China 6. N anjing University of Science and Technology, Nanjing, China 7. Lishui Hospital of Zhejiang University, Lishui, China # These authors contribute equally *Corresponding author A BSTRACT We propose a Hierarchical Multi-scale Knowledge-aware G raph Network (HMKGN) that models multi-scale interactions and spatially hierarchical relationships within whole-slide images (WSIs) for cancer prognostication. Unlike conventional attention-based MIL, which ignores spatial organization, or graph-based MIL, which relies on static handcrafted graphs, HMKGN enforces a hierarchical structure with spatial locality constraints, wherein local cellular-level dynamic graphs aggregate spatially proximate patches within each region of interest (ROI) and a global slide-level dynamic graph integrates ROI-level features into WSI-level representations. Moreover, multi-scale integration at the ROI level combines coarse contextual features from broader views with fine-grained structural representations from local patch-graph aggregation. We evaluate HMKGN on four TCGA cohorts (KIRC, LGG, PAAD, and STAD; N=513, 487, 138, and 370) for survival prediction. It consistently outperforms existing MIL-based models, yielding improved concordance indices (10.85% better) and statistically significant stratification of patient survival risk (log-rank p < 0.05). Index Terms Multiple instance learning, Multi-scale feature fusion, Hierarchical graph neural network, Knowledge-aware attention, Survival analysis 1. INTRODUCTION The rapid development of digital pathology has made whole- slide images (WSIs) a crucial foundation for computer- assisted tumor diagnosis, subtyping, and prognosis assessment[1]. WSIs contain rich morphological information across scales, simultaneously capturing cellular-level morphology and slide-level architecture, which are inherently complementary for survival analysis[2]. Accurate survival prediction plays a crucial role in clinical decision-making by guiding personalized treatment planning, assessing recurrence risk, and predicting patient prognosis[3]. There is a recent surge of multi-instance learning (MIL) in computational pathology. MIL-based approaches have achieved remarkable progress in slide-level classification and survival prediction, as they enable the end-to-end WSI-level modeling by learning the interaction and aggregation of individual patches across tissue volumes [4]. Many attention- based MIL approaches, such as TransMIL, are capable of effectively aggregating an arbitrary number of patches into a WSI-level representation [5], [6]. However, these methods treat individual patches as spatially irrelevant instances, neglecting the inherent spatial arrangement and morphological continuity across nearby patches. As a result, the derived slide-level representations may overlook key spatial context and structural dependencies that are critical for capturing tissue organization and pathological patterns. Meanwhile, graph-based MIL explicitly encodes spatial relationships among patches using graph structures. Yet, most existing methods depend on static, handcrafted graphs defined purely by spatial proximity, which restricts their ability to model broader contextual relationships [7]. Recent advances, such as the knowledge-guided WiKG framework, on the other hand, have demonstrated the benefit of dynamically learning contextual dependencies between patches[8]. Nevertheless, WiKG remains limited by the absence of spatial locality constraints, which permits interactions between distantly located regions that are less likely to be biologically related. Moreover, it operates on single-scale data, which restricts its ability to capture hierarchical and multi-scale contextual information, a Published as a conference paper at ISBI 2026 limitation in diseases characterized by spatial heterogeneity (e.g., pancreatic cancers [9]). To address these limitations, we propose a Hierarchical Multi-scale Knowledge-aware Graph Network (HMKGN) for WSI-based survival analysis. Our core innovations are: (1) spatial locality-constrained hierarchical learning, which aggregates spatially proximate patches into ROI-level representations and hierarchically integrates them into slide- level features to reflect tissue organization. (2) multi-scale feature fusion, which combines coarse contextual representations from lower-magnification views with fine- grained morphological information from high-magnification patches, enabling comprehensive modeling of both regional tissue context and cellular-level structure [10]. 2. M ETHOD 2.1. Data Preparation As shown in Fig. 1-a, the tissue regions in WSIs were first divided into non-overlapping patches of size at low magnification (5×). Each 5× patch was spatially aligned with its corresponding high-magnification region , which was further subdivided into a grid of 20× patches . Specifically, i ndexes the WSIs in the dataset, indexes the low-magnification patches within the -th WSI, and indexes the high-magnification patches corresponding to each low-magnification patch. Both low- and high-magnification patches were then embedded into feature vectors using a resolution-adaptive foundation model (UNIv2 [11]): (1) where denotes the encoder of the frozen UNIv2 model. This explicit multi-scale correspondence ensures pixel- level alignment between and , providing a geometric constraint for subsequent multi-scale feature integration. 2.2. Spatial Locality-Constrained Hierarchical Learning To capture the hierarchical structure of high-magnification p atches with spatial locality constraints, HMKGN comprises two stages: (1) local aggregation (patch-to-ROI) and (2) global aggregation (ROI-to-WSI). During the local aggregation stage (Fig. 1-b1), a spatial locality constraint was applied such that only patches within a grid were directly connected through a local dynamic graph . For each high magnification patch node feature in , its fine-grained ROI-level representation was obtained via: (2) which captures fine-grained cellular and morphological semantics within spatially coherent local regions. Each r epresentation was then fused with the coarse contextual representation from its spatially aligned region, yielding the ROI-level embedding (section 2.3). During the global aggregation stage (Fig. 1-b2), all ROI- level embeddings from local dynamic graphs () were integrated through a global dynamic graph to derive the slide-level representation: , (3) This hierarchical message passing jointly models local dependencies and global tissue context. The aggregation function Agg here corresponds to the knowledge-aware dynamic graph network (KGN) described in Section 2.4, which we build upon and extend with spatial locality constraints and multi-scale fusion. 2.3. Multi-scale Feature Fusion As shown in Fig. 1-c , for each ROI, coarse contextual representations derived from lower-resolution ROI images were fused with fine-grained features extracted from high-resolution patch graphs through a bidirectional cross-attention mechanism (BiX) [12]. Specifically, the queries (Q), keys (K), and values (V) were defined as (4) The cross-attention in both directions was computed as (5) and the fused regional representation was obtained by concatenating the two attention outputs: (6) This multi-scale integration enables the model to jointly capture both coarse regional tissue contextual semantics (low magnification) and fine-grained cellular-level morphology (high magnification) within a unified ROI representation. 2.4. Knowledge-aware attention-based Dynamic Graph All patch-level features are treated as nodes in the ROI-level patch graph during the local aggregation. All ROI-level features (with multi-scale feature fusion) are treated as nodes in the slide-level graph d uring the global aggregation. Message passing across this graph captures long-range structural dependencies and integrates regional information into a global slide representation . The final WSI-level feature vector p rovides a coherent multi-scale embedding suitable for Published as a conference paper at ISBI 2026 downstream tasks such as classification, molecular subtyping, and survival prediction. While the ROI-level graph module is inspired by knowledge-aware graph modeling techniques (e.g., WiKG), our framework extends this foundation by introducing the spatial locality constrained hierarchical structure and multi-scale feature fusion, enabling explicit modeling of multi-scale interactions and improving representation quality for WSI analysis. Fig. 1. Framework of the Proposed HMKGN Method. 2.5. Slide-level Survival Prediction with Hierarchical Modeling and Multi-scale Fusion The output aggregated slide-level features are used to predict survival end-to-end with a discrete-time negative log-likelihood survival loss (NLLSurvLoss). Importantly, slide-level predictions are informed by the hierarchical multi- scale graph, where fine-grained cellular-level features and coarse regional tissue contextual semantics features are aggregated under spatial locality constraints. The BiX effectively integrates the fine-grained high-magnification information with coarse low-magnification context, ensuring that both local morphology and global tissue architecture contribute to survival modeling. This multi-scale fusion preserves cross-level semantic consistency and enhances the - and long-range prognostic dependencies, providing a robust foundation for WSI-based survival prediction. 3. RESULT AND DISCUSSION We evaluated the proposed HMKGN against six established multiple-instance learning (MIL) and graph neural network (GNN)-based methods, including MaxMIL, MeanMIL, DAttention[5], TransMIL[6], MambaMIL[13], and WiKG[8], on four TCGA datasets (KIRC, LGG, PAAD, and STAD). Table 1 summarizes the concordance index (C-index) of each model, reported as mean ± standard deviation (SD) based on 4-fold cross-validation. HMKGN consistently outperforms existing methods across all cancer types and achieves 4.20% to 33.56% higher C-indices on the four datasets. These results demonstrate the efficacy of integrating multi-scale hierarchical features and BiX with knowledge- aware graph attention for WSI survival analysis. Table 1. Performance comparison of HMKGN and baseline models on TCGA datasets (C-index ± SD). (*) denotes significant logrank p-values. ModelKIRC LGGPAAD STAD Mean Improve(%) MaxMIL 0.7273 ± 0.0390(*) 0.6148 ± 0.0377(*) 0.5743 ±0.0730 0.5293 ± 0.0493(*) 16.50 MeanMIL 0.7131 ± 0.0253(*) 0.7549 ± 0.0509(*) 0.6095 ± 0.0744(*) 0.6152 ± 0.0908(*) 5.82 DAttention 0.7266 ± 0.0282(*) 0.7578 ± 0.0457(*) 0.5949 ± 0.0803 0.6191 ± 0.0869(*) 5.59 TransMIL 0.5241 ± 0.0563 0.5685 ± 0.0450 0.4828 ± 0.0533 0.5579 ± 0.0336 33.56 MambaMIL 0.7192 ± 0.0254(*) 0.7623 ± 0.0577(*) 0.6146 ± 0.0382(*) 0.6384 ± 0.0565(*) 4.20 WiKG 0.7176 ± 0.0379(*) 0.7648 ± 0.0449(*) 0.6185 ± 0.0690(*) 0.6170 ± 0.0552(*) 4.83 HMKGN (Our Model) 0.7486 ± 0.0298(*) 0.7699 ± 0.0522(*) 0.6897 ± 0.0723(*) 0.6411 ± 0.0565(*) As shown in Table 2, to evaluate the contribution of each component, we performed ablation studies on three key modules hierarchical modeling, multi-scale interaction, and spatial locality constraint. Table 2. Ablation study evaluating the effects of hierarchical modeling, multi-scale interaction, and spatial locality constraint across TCGA cohorts. (*) denotes significant logrank p-values. ModelHierarch .Locality Multi- scale KIRC LGG PAAD STAD KGN (WiKG)noN/Ano 0.7176 ± 0.0379(*) 0.7648 ± 0.0449(*) 0.6185 ± 0.0690(*) 0.6170 ± 0.0552(*) Published as a conference paper at ISBI 2026 Model Hierarch . Locality Multi- scale KIRC LGG PAAD STAD HMKGN single-scale yes yes no (20x only) 0.7446 ± 0.0217(*) 0.7657 ± 0.0456(*) 0.6327 ± 0.0404(*) 0.6390 ± 0.0741(*) HMKGN no_locality yes no N/A 0.7377± 0.0316 (*) 0.7584± 0.0309(*) 0.6341± 0.0611(*) 0.5984± 0.0627(*) HMKGNyes yes yes 0.7486 ± 0.0298(*) 0.7699 ± 0.0522(*) 0.6897 ± 0.0723(*) 0.6411 ± 0.0565(*) W ith only the hierarchical modeling, HMKGN single-scale solely utilizes to represent , where is obtained by aggregating high-resolution patch features within each ROI region, without incorporating the low-resolution patch features . Despite operating on a single scale, HMKGN single-scale outperforms the KGN baseline across KIRC, PAAD, and STAD (+2.2% to 3.6% on average), while performing comparably in LGG. This confirms that explicitly modeling region-level spatial hierarchy enables the network to aggregate semantically related patches and capture broader tissue context. With both hierarchical modeling and bidirectional multi- scale fusion present, HMKGN further combines the low- resolution features with the high-resolution aggregated representation and outperforms HMKGN single-scale (by 0.3% to 9%), with the largest gain observed in PAAD. This likely reflects the pronounced spatial heterogeneity of pancreatic adenocarcinoma in PAAD [9], which can be captured by multi-scale learning through integration of fine- grained morphological details and coarse contextual structures. When the spatial locality constraint was removed (HMKGN no _ locality ), the model exhibited a consistent decline (-1.5% to -8.8%) across all cohorts compared to HMKGN, especially in PAAD (-8.8%) and STAD (-7.1%). Therefore, preserving local spatial order among high-magnification patches is essential for capturing prognostically relevant micro-architectural patterns. Overall, our approach (HMKGN) achieves the best C- index and statistically significant log-rank p-values (p < 0.05). This demonstrates that the combination of hierarchical structure, reciprocal multi-scale fusion, and spatial locality constraint leads to a more discriminative and biologically meaningful multi-scale representation for survival prediction performance across all TCGA cohorts. 4. CONCLUSION In this study, we introduced HMKGN, a hierarchical graph framework that unifies spatial locality constraints, hierarchical aggregation, and feature fusion to enhance multi- scale representation learning across different magnifications in WSIs. The proposed model achieved superior survival prediction performance across four TCGA datasets with over 1500 patients. These results highlight the strong potential of HMKGN for clinical translation, particularly in improving prognostic assessment. In future work, we plan to extend HMKGN to multi-modal survival analysis by integrating pathological, radiological, and clinical data. 5. ACKNOWLEDGMENTS This work is made possible by the National Natural Science Foundation of China under award numbers 62301265. The authors have no relevant financial or non-financial interests to disclose. 6. REFERENCES [ 1] G. Litjens et al. Med. Image Anal., vol. 42, p. 6088, 2017. [2] R. J. Chen et al. images via hierarchical self- Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, p. 16144 16155. [3] K. Bera, K. A. Schalper, D. L. Rimm, V. Velcheti, and A. Nat. Rev. Clin. Oncol., vol. 16, no. 11, p. 703 715, 2019. [4] P. Sandarenu et al. breast cancer using multiple instance learning of Sci. Rep., vol. 12, no. 1, p. 14527, 2022. [5] -based deep International conference on machine learning, PMLR, 2018, p. 2127 2136. [6] Z. Shao et al. multiple instance learning for whole slide image Adv. Neural Inf. Process. Syst., vol. 34, p. 2136 2147, 2021. [7] S. Javed et al. Med. Image Anal., vol. 63, p. 101696, 2020 . [ 8] J. Li et al. - aware attention for histopathology whole slide image Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, p. 11323 11332. [9] D. Cui Zhou et al. transitional cell populations cooperate with the microenvironment in untreated and chemo-resistant Nat. Genet., vol. 54, no. 9, p. 1390 1405, Sep. 2022, doi: 10.1038/s41588-022-01157-1. [10] S. Diao et al. -magnification similarity learning IEEE J. Biomed. Health Inform., vol. 27, no. 3, p. 1535 1545, 2023. [11] R. J. Chen et al. -Purpose Foundation Nat. Med., 2024. [12] bidirectional cross-ArXiv Prepr. ArXiv220105887, 2022. [13] long sequence modeling with sequence reordering in International conference on medical image computing and computer-assisted intervention, Springer, 2024, p. 296 306. Published as a conference paper at ISBI 2026