Paper deep dive

GateSPINE: Gated Cross-View Fusion for Lumbar Spine MRI Report Generation

Hoang Nguyen Van, Cuong Vuong Tuan, Trang Mai Xuan, Bien Tran Van, Nam Tran Van, Thien Van Luong

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 39

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Automated report generation can ease the burden radiolo gists face when interpreting multi-sequence MRI studies. Unlike CT, MRI examinations comprise multiple sequences and imaging planes, each con tributing complementary diagnostic information. Existing methods en code a study as a single volume and combine multiple acquisitions by fixed rules. Findings visible in only one plane are thus diluted and of ten missed, lowering recall on clinical efficacy metrics, where a missed abnormality is most costly. We propose GateSPINE, a vision-language framework that fuses sagittal T1 and T2 volumes with a training-free operator, encodes the fused sagittal and axial volumes with two parallel 3D encoders, and decodes their combined representation into a report. Its core mechanism is a gated cross view fusion module that predicts, per feature channel and token, how much of each view to admit, so the more informative view dominates at each spatial location. We evaluate GateSPINE on three lumbar MRI datasets, comprising two public bench marks and a private cohort collected from Phenikaa University Hospital, using both natural language generation (NLG) and clinical efficacy (CE) metrics. GateSPINE achieves the highest CE F1 through improved re call on all three datasets; on SPIDER, which lacks an axial sequence, this reflects the sagittal fusion component rather than the gated cross-view mechanism, which is validated on the two cohorts with both imaging planes. GateSPINE also remains competitive on standard NLG metrics.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

36,042 characters extracted from source content.

Expand or collapse full text

GateSPINE: Gated Cross-View Fusion for Lumbar Spine MRI Report Generation Hoang Nguyen Van 1 , Cuong Vuong Tuan 1 , Trang Mai Xuan 1⋆ , Bien Tran Van 2,3 , Nam Tran Van 3 , and Thien Van Luong 4 1 Applied AI Lab, Phenikaa University, Hanoi, Vietnam 2 Medical Imaging & Radiological Technology Department, Faculty of Medical Technology, Phenikaa School of Medicine & Pharmacy, Phenikaa University, Vietnam 3 Radiology & Functional Exploration Center, Phenikaa University Hospital, Vietnam 4 Business AI Lab, College of Technology, National Economics University, Vietnam 23010101@st.phenikaa-uni.edu.vn, cuong.vuongtuan, trang.maixuan, bien.tranvan@phenikaa-uni.edu.vn, bv.namtv@phenikaamec.vn, thienlv@neu.edu.vn Abstract. Automated report generation can ease the burden radiolo- gists face when interpreting multi-sequence MRI studies. Unlike CT, MRI examinations comprise multiple sequences and imaging planes, each con- tributing complementary diagnostic information. Existing methods en- code a study as a single volume and combine multiple acquisitions by fixed rules. Findings visible in only one plane are thus diluted and of- ten missed, lowering recall on clinical efficacy metrics, where a missed abnormality is most costly. We propose GateSPINE, a vision-language framework that fuses sagittal T1 and T2 volumes with a training-free operator, encodes the fused sagittal and axial volumes with two parallel 3D encoders, and decodes their combined representation into a report. Its core mechanism is a gated cross-view fusion module that predicts, per feature channel and token, how much of each view to admit, so the more informative view dominates at each spatial location. We evaluate GateSPINE on three lumbar MRI datasets, comprising two public bench- marks and a private cohort collected from Phenikaa University Hospital, using both natural language generation (NLG) and clinical efficacy (CE) metrics. GateSPINE achieves the highest CE F1 through improved re- call on all three datasets; on SPIDER, which lacks an axial sequence, this reflects the sagittal fusion component rather than the gated cross-view mechanism, which is validated on the two cohorts with both imaging planes. GateSPINE also remains competitive on standard NLG metrics. Keywords: Lumbar spine MRI· Medical report generation· 3D vision encoders· Gated cross-wiew fusion· Clinical efficacy. ⋆ Corresponding author. arXiv:2609.40091v1 [cs.CV] 30 Sep 2026 2H. V. Nguyen et al. 1 Introduction Magnetic resonance imaging (MRI) is widely used to evaluate degenerative lum- bar spine diseases because it provides detailed visualization of soft tissues and neural structures [1, 14]. In clinical practice, radiologists must examine mul- tiple MRI sequences (e.g., T1W and T2W) acquired from different imaging planes (e.g., sagittal and axial), integrate the complementary findings across these images, and produce a comprehensive diagnostic report. This process is time-consuming because it requires careful interpretation of a large volume of imaging data, so automatic radiology report generation has attracted increasing attention [4,11,12]. Automatic report generation from CT typically couples a 3D vision encoder with a language decoder [11]. Most recently, Reg2RG [7] grounds the report on anatomical regions and combines region-level with global features, substan- tially improving diagnostic accuracy. These methods assume that a study is a single volume, whereas an MRI examination comprises several sequences and imaging planes, each showing some findings more clearly than the others. For lumbar spine MRI specifically, SPINE [12] stacks sagittal T1- and T2-weighted images together with segmentation labels, showing that integrating complemen- tary contrasts improves report quality. Yet it models the sagittal view alone. This omission is consequential because lateral recess stenosis, facet hypertrophy, and the lateralization of disc herniation are assessed on axial images, findings that a sagittal-only model can therefore characterize only in part. CrossSpine [16] attends across sequences to predict degeneration grades, but represents each by a single 2D slice. The two views are not interchangeable. Sagittal images trace continuity along the spinal axis and capture disc height, alignment, and marrow signal across all levels at once, whereas axial images expose a per-level geometry that sagittal slices cannot reconstruct. Which view carries the decisive evidence therefore changes from finding to finding, which a fixed combination rule cannot accommodate. To our knowledge, learning to combine sagittal and axial views for lumbar report generation remains unexplored. To address this gap, we propose GateSPINE, a unified vision–language framework for lumbar MRI report generation. GateSPINE first merges sagittal T1 and T2 with a training-free dynamic fusion operator, then encodes the fused sagittal and the axial volumes with two parallel 3D vision encoders. A lightweight gated cross-view fusion module learns, per feature channel, how much of each view to retain, and the resulting representation is decoded into a report by a vision–language model. In summary, our contributions are as follows: – We propose GateSPINE, a unified vision–language framework that inte- grates multi-sequence fusion, parallel 3D vision encoders, and adaptive cross- view fusion, improving both clinical-efficacy and language-generation perfor- mance. – We design a lightweight gated cross-view fusion module that adaptively com- bines sagittal and axial representations. The module learns to emphasize the view that provides more informative evidence for each feature channel and Gated Cross-View Fusion for Lumbar Spine MRI Report Generation3 spatial position, leading to more effective use of complementary information across MRI sequences and imaging planes. – On the private PhenikaaMec cohort and two public datasets (SPIDER, Lum- bar), GateSPINE improves clinical-efficacy micro-F1 over the strongest base- line by 3.6, 3.6, and 26.9 points, driven by recall gains of up to 38.0 points; on SPIDER, which lacks an axial sequence, this reflects the sagittal fusion component rather than the gated cross-view mechanism. GateSPINE also achieves the highest BERTScore on all three datasets (+0.3, +1.3, and +0.4 points). 2 Related Work CT report generation. Most 3D report generation systems extract volume features with a 3D vision encoder and pass them to a language decoder that writes the report. CT2Rep [11] is an early instance, though its decoder is trained from scratch, which limits fluency. Dia-LLaMA [8] replaces it with a pretrained LLM and prompts it with predicted disease labels, though it depends on an aux- iliary classifier for those labels. Med3DVLM [22] pursues a different axis, merging low- and high-level features to cut encoding cost, while generalist models treat report generation as one of many tasks [4, 21]. Each of these systems therefore combines more than one feature stream. Yet all those streams are extracted from a single volume, so combining them never involves reconciling different views of the anatomy. MRI report generation. Report generation for MRI must contend with mul- tiple sequences and imaging planes within one examination. SPINE [12] stacks sagittal T1 and T2 channel-wise with segmentation-derived channels, showing that combining contrasts improves report quality. Such stacking relies on both sequences being acquired in the same plane, so it does not extend to axial im- ages, whose evidence the model never sees. Other systems take a different route, working from structured gradings [18] or radiologist-authored findings [23] rather than from the images themselves. How to combine sagittal and axial evidence therefore remains open. We address this by fusing sequences within a plane and gating across planes at the feature level. 3 Methods The overview of our method GateSPINE is shown in Fig. 1. Lumbar MRI report generation involves creating a report R from a study of three volumes: sagittal T1-weighted V T1 , sagittal T2-weighted V T2 , and axial T2-weighted V ax , each inR D×H×W . Unlike previous works [11, 21, 22] that encode a study as a single volume, we preserve the two imaging planes as separate streams and let the model decide, at each location, which to trust. 4H. V. Nguyen et al. Test time dynamic fusion 1024 Vision Encoder 24 32p x 256p x 15 1024 1024 32p x 256p x 256p x 256p x 256 256p x 32p x 256p x 256p x 32p x Vision Encoder 256p x 3D Projection ... Large Language Model Report FrozenFine-tuned C 1024 <start_of_turn>user [<im_patch>x256] Describe the abnormalities in this spinal MRI <end_of_turn> <start_of_turn>model ◉ ◉ + (2) Gated Cross-View Fusion (1) Multi-Sequence Encoder C Concatenate FC + Sigmoid ◉ Element-wise multiplication + Element-wise addition Fig. 1. Overview of the GateSPINE framework. Sagittal T1 and T2 volumes are fused by the TTD module, while the axial T2 volume is processed in parallel. Two ViT3D encoders extract features from the fused sagittal volume and the axial volume, and the Gated Cross-View Fusion layer combines them. The 3D Projector then compresses the fused features into visual tokens for the VLM decoder, which generates the final report. Since the sagittal sequences share a common geometry, we merge them into ̃ V sag = φ(V T1 ,V T2 ) with test-time dynamic fusion (TTD) [6], a training-free adaptive fusion operator. Two parallel ViT3D encoders [9] f sag and f ax , initial- ized from M3D [4], then extract view-specific features. As the two planes may conflict locally, we propose a gated cross-view fusion module f G that predicts, per channel and per position, how much of each view to admit. A projector f P compresses the fused features into visual tokens, which the MedGemma de- coder [19] consumes with a prompt P to generate R: R = LLM f P f G (F sag ,F ax ) , P ,(1) where F sag = f sag ( ̃ V sag ) and F ax = f ax (V ax ). 3.1 Multi-Sequence Encoder This stage maps the three input volumes to two view-level representations F sag and F ax : the two sagittal sequences, which share an acquisition geometry, are merged before encoding, while the axial volume forms its own stream. Sagittal fusion. T1w and T2w are complementary, showing morphology and marrow signal versus fluid and edema. Since a fixed rule cannot follow which contrast is locally stronger, we merge them with TTD [6], a training-free operator built on CDDFuse [25]. “Test-time” refers to the operator itself, which adapts its weights to each input without being trained on our data; we apply it identically Gated Cross-View Fusion for Lumbar Spine MRI Report Generation5 to every study, as a fixed preprocessing step, both when training GateSPINE and at inference. Slice by slice, a shared pretrained autoencoder reconstructs each source and measures its per-pixel squared error ℓ s (i,j), from which a Relative Dominability weight follows: z s (i,j) = exp − γ ℓ s (i,j) ,(2) w s (i,j) = α exp z s (i,j) P s ′ ∈S exp z s ′ (i,j) , s∈S,(3) where γ > 0 sets how sharply the weighting responds to error, and scaling by α =|S| makes the weights average one. A source reconstructing a location well is thus emphasized while both are retained. The weighted features are decoded into a fused slice, and stacking all slices gives ̃ V sag = φ(V T1 ,V T2 ). View encoding. Both ̃ V sag and V ax are min–max normalized and resampled to a common resolution (D t ,H t ,W t ), then encoded by two ViT3D encoders [9] f sag and f ax with separate weights, initialized from M3D [4]. Each partitions its input into non-overlapping patches of size (p D ,p H ,p W ) and emits N tokens of dimension d, so that F sag ,F ax ∈R N×d . The axial branch requires no sequence fusion, as a single axial sequence is acquired. 3.2 Gated Cross-View Fusion This module takes the two view representations from Sec. 3.1 and produces a single representation F fuse ∈R N×d in which each view contributes according to its content. Concatenating the two token sets along the sequence would double their number to 2N and raise the decoder cost accordingly, while averaging them applies the same proportion at every channel and location. We instead combine them along the feature dimension with a learned gate, which leaves the token count unchanged and lets the mixing proportion vary with content. Following the gated fusion formulation of [3], but predicting a gate per channel and per token rather than one scalar per modality, we compute for each token n∈1,...,N g n = σ W g [f sag n ; f ax n ] + b g ∈R d ,(4) f fuse n = g n ⊙ f sag n + (1− g n )⊙ f ax n ,(5) where f sag n ,f ax n ∈R d are the n-th rows ofF sag andF ax , [· ; · ]∈R 2d denotes vec- tor concatenation, W g ∈R d×2d and b g ∈R d are learnable, σ(·) is the sigmoid, and ⊙ is element-wise multiplication. Stacking over n gives F fuse , at a cost of 2d 2 + d parameters. Because g n is d-dimensional, different channels may be drawn from different views within the same token, and the gate is conditioned on both views jointly, so the proportion given to one depends on what the other contains. The alternatives are recovered as special cases: a constant g n ≡ 1 2 gives averaging, and a gate held fixed across tokens reduces to one scalar weight per view. 6H. V. Nguyen et al. 3.3 Vision–Language Projection and Decoding This stage compressesF fuse into visual tokens the language model can consume. We reshape it into its 3D patch grid and apply non-overlapping average pooling with factor ρ per axis, giving N v = N/ρ 3 tokens F seq ∈R N v ×d while preserving spatial structure. Following the projection design of the pretrained VLM [19], a trainable adapter maps these into its d m -dimensional space, after which its own frozen RMSNorm and projection W p are applied: F vis = W p RMSNorm GELU(W a F seq + b a ) ∈R N v ×d LLM .(6) Training only the adapter (W a ,b a ) reuses the pretrained vision–language align- ment. The visual tokens then replace the placeholder embeddings in P, forming a multimodal sequence of length M = 1 +N v +m from a BOS token, N v visual tokens, and m prompt tokens. We adapt the decoder with LoRA [13] on all linear layers, while the view encoders, the gate, and the adapter are trained with full gradients. 3.4 Training and Inference The model is trained with an autoregressive cross-entropy loss over report tokens R =r 1 ,...,r T , conditioned on the multimodal input: L =− T X t=1 μ t logp θ r t | r <t ,F vis ,P , μ t = 1[r t is an answer token ], (7) where μ t restricts the gradient to answer tokens, so the model is not trained to reproduce the prompt. During training the prompt is sampled at random from a set of clinical query templates, improving robustness to prompt wording. At inference, reports are decoded greedily, stopping at the end-of-sequence token or after T max tokens. 4 Experiments and Results 4.1 Datasets and Evaluation Metrics Datasets. PhenikaaMec [20] is an internal cohort of 250 patients (112/138 M/F, aged 48.1 ± 15.2 years) with sagittal T1, sagittal T2, and axial T2 at 0.3× 0.3× 4.4 m, extracted from hospital PACS and anonymized under ap- propriate ethical clearance; 236 remain after discarding cases lacking a report or axial sequence. The reports are translated from Vietnamese with LLaMA- 3.3-70B, and we split by acquisition month into 118/58/60 studies training on earlier months and testing on later ones to mirror deployment. Lumbar [2] con- tains 575 Mendeley cases (one per patient) with both planes; 515 remain after discarding empty reports, split 70:15:15. SPIDER [10] provides sagittal T1, T2, and segmentation masks but no axial sequence and no free-text reports, split Gated Cross-View Fusion for Lumbar Spine MRI Report Generation7 60:20:20 by patient. On SPIDER, all methods use the reference reports released by SPINE [12], generated from grading labels with GPT-4. Without an axial sequence the cross-view module is inactive, and, following SPINE, the mask is combined with T1 and T2 per slice in ratio 2 : 2 : 3 in place of TTD; SPIDER thus tests the sagittal branch, while the gated cross-view module is evaluated on the two cohorts with both planes. Evaluation Metrics. Following recent report generation work [7,12], we report language quality (NLP) and clinical efficacy (CE) metrics. The NLP metrics are BLEU-4 [17], ROUGE-1, ROUGE-L [15], METEOR [5], and BERTScore [24]. These cannot capture diagnostic correctness, since a negation flips meaning while leaving surface similarity intact. CE metrics therefore compare clinical entities: an LLM judge (LLaMA-3.3-70B) labels nine lumbar findings—disc herniation, bulge, degeneration, spinal stenosis, osteophytes, nerve compression, curvature abnormality, annular tear, and disc height loss—as present or absent in both generated and reference reports. We report Micro-F1 pooled over findings and Macro-F1 averaged over them. The same judge scores every method, so the com- parison is consistent across models; validating the judge and the report transla- tion against radiologist annotation is left to future work. 4.2 Implementation Details and Baselines Implementation. GateSPINE is implemented in PyTorch and trained on one NVIDIA Quadro RTX 8000, with MedGemma 4B-IT as decoder and both ViT3D encoders initialized from M3D [4]. Volumes are resampled to D t × H t × W t = 32×256×256 and split into (4, 16, 16) patches (N = 2048, d = 768); pooling with ρ = 2 gives N v = 256 visual tokens, projected via d m = 1152 to d LLM = 2560, and TTD uses γ = 50. Stage 1 trains the encoders, gate, and adapter with the LLM frozen (learning rate 10 −4 ); stage 2 adds LoRA [13] (rank 8, scaling 32, dropout 0.05) and fine-tunes jointly with a per-dataset learning rate. Both stages use AdamW with cosine decay, mixed precision, an effective batch size of 8 via gradient accumulation, and 10–15 epochs with early stopping. Baselines. We compare against CT2Rep [11], Med3DVLM [22], Dia-LLaMA [8], and SPINE [12], retrained on each dataset with the same preprocessed inputs and greedy decoding. Being single-volume methods, they receive the fused sagittal volume without the axial branch, so this comparison reflects the framework as a whole; the ablation in Sec. 4.4, where every variant receives both planes, isolates the fusion mechanism. 4.3 Evaluation Results On PhenikaaMec, GateSPINE achieves the best clinical entity scores and leads on ROUGE and BERTScore (Table 1). Its clinical gain comes mainly from higher recall (Micro-R 67.35 vs. 52.05 for Dia-LLaMA) at comparable precision, so it misses fewer true findings. Dia-LLaMA scores higher on BLEU-4 and METEOR but is much weaker on clinical entities. Across the nine findings (Fig. 2), Gate- SPINE leads on seven of nine, with the clearest margins over SPINE on disc 8H. V. Nguyen et al. Table 1. Comparison with state-of-the-art methods on language-quality (NLP) and clinical-efficacy (CE) metrics across three datasets. Best and second-best results in bold and underlined . Method NLPCE MicroCE Macro BL-4 RG-1 RG-L MTR BERTPre. Rec. F1Pre. Rec. F1 PhenikaaMec CT2Rep [11]25.91 57.97 44.21 45.58 89.7866.67 2.37 4.5861.11 2.08 3.99 Dia-LLaMA [8] 28.16 58.9945.0550.57 90.8874.7952.05 61.3871.79 47.75 53.55 Med3DVLM [22]4.35 36.99 20.27 22.78 87.0336.51 6.69 11.3028.86 6.68 10.19 SPINE [12]25.04 56.84 42.71 45.99 90.4371.24 64.0267.4965.23 57.1758.78 GateSPINE26.4660.24 46.15 48.6391.1675.33 67.35 71.1265.3059.69 60.50 SPIDER CT2Rep [11]62.76 77.1377.3787.4594.8387.50 50.60 64.1259.17 40.46 47.24 Dia-LLaMA [8]53.79 64.70 63.59 64.35 93.6589.4776.1282.2670.9961.6764.98 Med3DVLM [22]45.87 59.81 58.69 57.79 93.0693.55 68.24 78.9184.96 56.61 64.16 SPINE [12]63.2071.10 70.70 77.40 94.8066.90 64.00 64.9049.30 49.20 46.70 GateSPINE65.67 79.39 79.27 87.50 96.0983.91 87.95 85.8870.49 74.36 72.21 Lumbar CT2Rep [11]20.4249.6747.7647.4090.2055.66 31.2240.0013.61 11.14 12.19 Dia-LLaMA [8]8.49 33.74 30.81 29.95 87.3256.62 8.72 15.1215.40 3.29 5.40 Med3DVLM [22] 9.04 34.38 29.38 28.52 86.9958.55 30.96 40.5330.46 17.54 21.87 SPINE [12]12.72 40.45 36.63 35.64 88.9468.5430.81 42.5145.26 19.2424.18 GateSPINE23.73 51.95 48.65 48.07 90.6269.59 69.23 69.4137.6934.65 33.39 degeneration and osteophytes; it trails only on disc bulge, a naming bias rather than a miss: of 29 reference bulges it calls 24 a herniation and omits only 4. Since bulge and herniation are separate labels on a clinical continuum, this low- ers Macro-F1 more than Micro-F1. On SPIDER, GateSPINE leads on all NLG metrics and on both F1 scores, with the largest margin on Macro-F1 (+7.2). Med3DVLM attains higher precision but far lower recall, describing fewer find- ings overall. Because the references here are GPT-4-generated from grading la- bels rather than written by radiologists, they are highly templated; absolute scores on SPIDER are inflated relative to the other two datasets and should be read only within this benchmark. On the public Lumbar dataset, GateSPINE is again best on both metric groups, above CT2Rep and SPINE. Macro-F1 stays low because every model scores zero on osteophytes and disc height loss, and the short segmentation-style references make both metric groups harder to satisfy. 4.4 Ablation Studies and Analyses Contribution of MRI Sequences. We assess each sequence by turning se- quences on and off on PhenikaaMec and Lumbar, as shown in Table 2. A single sequence is not enough on either dataset. On PhenikaaMec, sagittal T2 is the best single view, since most degenerative findings are clearer on T2-weighted images, and sagittal T1 is the weakest. On Lumbar the order flips: axial T2 is strongest, matching a stenosis-focused cohort where the axial plane best shows canal narrowing. Adding the axial view on top of the fused sagittal T1 and T2 gives a large jump in clinical entity scores on both datasets. The full three- sequence setup is best on every metric, and the gated integration of the axial plane drives most of the clinical entity gain. Gated Cross-View Fusion for Lumbar Spine MRI Report Generation9 Disc herniation Disc bulge Disc degeneration Spinal stenosis Osteophytes Nerve compression Curvature abnormality Annular tear Disc height loss 20 40 60 80 100 GateSPINE (Ours) SPINE Dia-LLaMA CT2Rep Med3DVLM Fig. 2. F1 score for each clinical finding on PhenikaaMec. Table 2. Ablation on input MRI sequences. Best in bold, second best underlined . SequenceNLP MetricsCE (F1) DatasetT1 T2 AxBL-4 RG-1 RG-L MTR BERTMicro Macro PhenikaaMec ✓19.69 54.12 39.20 40.81 90.3031.12 17.53 ✓21.81 54.36 41.16 42.10 90.1856.2546.50 ✓22.11 55.6442.23 44.5390.2748.37 39.77 ✓23.5055.52 42.6343.90 90.4349.51 45.16 ✓26.46 60.24 46.15 48.63 91.1671.12 60.50 Lumbar ✓10.27 32.97 30.52 32.04 88.415.033.49 ✓20.91 43.03 38.80 42.60 90.4252.70 17.01 ✓21.0948.2244.1545.0689.7062.7625.79 ✓20.47 45.47 39.95 43.18 89.1554.04 26.44 ✓23.73 51.95 48.65 48.07 90.6269.41 33.39 Cross-View Fusion Mechanism. Table 3 compares three cross-view fusion methods. We first expected the uncertainty-aware dynamic fusion to perform best, but the simple gated fusion outperforms both concatenation and dynamic fusion. This holds on both PhenikaaMec and Lumbar. We attribute this to the gate acting as a stable, content-dependent selector that is easier to train on small medical data than a larger dynamic weighting module. LLM as the Language Decoder. To isolate the decoder’s role, we keep the 3D visual architecture, gated fusion, and projector fixed and swap only the LLM decoder, as shown in Table 3. MedGemma outperforms Phi-3 on every metric. The gap is small on language metrics but much larger on clinical efficacy. We attribute this to MedGemma’s biomedical pre-training, which turns the same features into more accurate findings. Analysis of Generated Reports. Figure 3 shows one representative case from each dataset, comparing generated and ground-truth reports and judged on ab- normality type, location, and severity. Across all three datasets, GateSPINE recovers most major abnormalities with fewer hallucinated and missed findings 10H. V. Nguyen et al. Table 3. Ablation on the cross-view fusion mechanism (rows 1–3, MedGemma decoder) and on the language decoder (rows 3–4, gate fusion; MedGemma 4B vs. Phi-3-mini 3.8B). Best in bold, second best underlined among fusion variants. DatasetFusionDecoder NLP MetricsCE (F1) BL-4 RG-1 RG-L MTR BERTMicro Macro PhenikaaMec Concatenation MedGemma23.61 56.8043.58 44.67 90.5953.11 46.65 DynamicMedGemma24.3756.60 44.3145.3790.7363.0153.46 GateMedGemma26.46 60.24 46.15 48.63 91.1671.12 60.50 GatePhi-3-mini20.94 53.47 39.28 41.51 89.9356.32 46.89 Lumbar Concatenation MedGemma18.56 44.25 40.79 40.56 89.5953.90 23.47 DynamicMedGemma21.1747.3543.0144.6789.6962.3232.42 GateMedGemma23.73 51.95 48.65 48.07 90.6269.41 33.39 GatePhi-3-mini18.35 47.10 43.45 43.08 89.7059.83 24.68 Ground-truth Lumbar spine curvature is decreased. Image of vertebral body osteophyte formation due to degeneration. No malignant lesions are seen in the vertebrae. No infectious pathology of the vertebral body-disc is seen. Posterior wall and pedicles are preserved. Lumbar intervertebral discs have normal height, with decreased signal on T2W due to degeneration. Disc protrusion at L3/4, L4/5 levels causing mild narrowing of the spinal canal and bilateral intervertebral foramina, currently not causing nerve root compression. Annular fissure of the L4/L5 disc. No space-occupying lesion is seen in the spinal canal. Conus medullaris, nerve roots, and cauda equina: no abnormality is seen. Both sacroiliac joints have normal signal. Ours Lumbar spine curvature is reduced. Image of vertebral body osteophyte formation. No malignant lesions are seen in the vertebrae. No infectious lesions are seen in the vertebral body or intervertebral disc. Posterior wall and pedicles are preserved. Lumbar intervertebral discs have normal height, decreased signal on T2W due to degeneration. Disc protrusion at L4/5, L5/S1 levels causing spinal canal stenosis and bilateral foraminal stenosis, the narrowest point is ~ 7.5m, compressing bilateral L5 nerve roots below the articular surface. No intraspinal mass is seen. Conus medullaris, nerve roots, and cauda equina: no abnormal mass is seen. Bilateral sacroiliac joints have normal signal. SPINE Lumbar spine physiological curvature is decreased. Lumbar vertebral bodies have uneven signals due to degeneration. No vertebral compression or spondylolisthesis is seen. Degenerative osteophytes are seen on lumbar vertebral bodies. Lumbar intervertebral discs have normal height. Uneven signal reduction on T2W due to degeneration. Annular fissure of L5/S1 intervertebral disc. Protrusion of L4/L5 and L5/S1 intervertebral discs causing spinal canal stenosis, with the narrowest anteroposterior diameter of approximately 10 m (at the level of L5/S1), resulting in compression of the left L5 nerve root. No abnormality is seen in the size and signal of the conus medullaris. No abnormality is seen in the soft tissues surrounding the vertebral bodies. No abnormality is seen in the bilateral sacroiliac joints. PhenikaaMec Feature of muscle spasm. wide base disc bulge noted at L5/S1 level, compressing the theca sac and both nerve roots . Diffuse disc bulge noted at L4/L5 level, mildly compressing the thecal sac and encroching exit neural canal."," LSS MRI : Feature of muscle spasm. Diffuse disc bulge noted at L4/L5 level, compressing the thecal sac and exit nerve roots" LSS MRI No evidence of disc herniation. No thecal sac or nerve root compression noted. There is evidence of degenerative changes across multiple lumbar levels with severe involvement at L2, L4, areas of disc bulging, associated Modic changes, endplate irregularities like Schmorl’s nodes, no signs of spinal instability. Special attention should be given to the levels L1-L2, L4-L5. There is evidence of degenerative changes across multiple lumbar levels with severe involvement at L1, L2, L3, L4, L5, areas of disc bulging, associated Modic changes, endplate irregularities like Schmorl’s nodes, no signs of spinal instability. Special attention should be given to the levels L1-L2, L4-L5. There is evidence of degenerative changes across multiple lumbar levels with severe involvement at L1, L2, L3, areas of disc bulging, associated Modic changes, endplate irregularities like Schmorl’s nodes, no signs of spinal instability. Special attention should be given to the levels L1-L2. Spider Lumbar Fig. 3. Generated report comparison: Ground-truth, Ours, and SPINE, one case per dataset. than SPINE, which both misses findings and adds irrelevant text. The remaining GateSPINE errors are concentrated in the vertebral level and in lesion severity, which we target in future work. 5 Conclusions In this paper, we propose GateSPINE, a unified vision–language framework for lumbar spine MRI report generation. Unlike existing methods that do not fully utilize the complementary information from different MRI views, GateSPINE introduces a lightweight gated cross-view fusion module to adaptively combine sagittal and axial representations. The proposed module learns how much infor- mation to preserve from each view for every feature channel, enabling the model to better exploit complementary information across MRI sequences and imag- ing planes. On one private and two public lumbar MRI datasets, GateSPINE achieves the best performance on clinical-efficacy metrics while remaining com- Gated Cross-View Fusion for Lumbar Spine MRI Report Generation11 petitive on language-generation metrics. Furthermore, ablation studies show that the gated cross-view fusion module outperforms existing fusion strategies. One limitation of the current framework is that it mainly relies on global image representations, which can cause the model to occasionally miss subtle lesions or describe their anatomical locations inaccurately, as illustrated in Fig. 3. In future work, we plan to incorporate finer-grained anatomical information, such as disc-level segmentation or lesion detection, to improve localization accuracy and further enhance report quality. Acknowledgements This research was partially funded by Vingroup Innovation Foundation (VINIF) under project code VINIF.2026.DA036. This research was also supported by the Ben Dam Me Award Fund, the Vietnam Young Talent Support Fund, and the Number One Brand, Tan Hiep Phat Group. References 1. Aaen, J., Austevoll, I.M., Hellum, C., Storheim, K., Myklebust, T.A., Banitalebi, H., Anvar, M., Brox, J.I., Weber, C., Solberg, T., Grundnes, O., Brisby, H., In- drekvam, K., Hermansen, E.: Clinical and MRI findings in lumbar spinal stenosis: baseline data from the NORDSTEN study. Eur Spine J 31(6), 1391–1398 (2022) 2. Al-Kafri, A.S., Sudirman, S., Hussain, A., Al-Jumeily, D., Natalia, F., Meidia, H., Afriliana, N., Al-Rashdan, W., Bashtawi, M., Al-Jumaily, M.: Boundary de- lineation of MRI images for lumbar spinal stenosis detection through semantic segmentation using deep neural networks. IEEE Access 7, 43487–43501 (2019) 3. Arevalo, J., Solorio, T., Montes-y-Gómez, M., González, F.A.: Gated Multimodal Units for Information Fusion. In: International Conference on Learning Represen- tations (2017) 4. Bai, F., Du, Y., Huang, T., Meng, M.Q.H., Zhao, B.: M3D: Advancing 3D medical image analysis with multi-modal large language models. arXiv:2404.00578 (2024) 5. Banerjee, S., Lavie, A.: METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. p. 65–72 (2005) 6. Cao, B., Xia, Y., Ding, Y., Zhang, C., Hu, Q.: Test-time dynamic image fusion. In: Advances in Neural Information Processing Systems. vol. 37, p. 2080–2105 (2024) 7. Chen, Z., Bie, Y., Jin, H., Chen, H.: Large language model with region-guided referring and grounding for CT report generation. IEEE Transactions on Medical Imaging 44(8), 3139–3150 (2025) 8. Chen, Z., Luo, L., Bie, Y., Chen, H.: Dia-LLaMA: Towards large language model- driven CT report generation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2025. Lecture Notes in Computer Science, vol. 15966, p. 141–151 (2025) 9. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (ICLR) (2021) 12H. V. Nguyen et al. 10. van der Graaf, J.W., van Hooff, M.L., Buckens, C.F.M., Rutten, M., van Susante, J.L.C., Kroeze, R.J., De Kleuver, M., van Ginneken, B., Lessmann, N.: Lumbar spine segmentation in MR images: a dataset and a public benchmark. Scientific Data 11(1), 264 (2024) 11. Hamamci, I.E., Er, S., Menze, B.: CT2Rep: Automated radiology report generation for 3D medical imaging. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. Lecture Notes in Computer Science, vol. 15012, p. 476–486 (2024) 12. Helmy, H., Hosseini, A., Ibrahim, A., Baig-Mirza, A., Sadek, A.R., Serag, A.: SPINE: Segmentation-guided processing and integration of multimodal spinal MRI for natural language enhanced report generation. Applied Artificial Intelligence (2026) 13. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Chen, W.: LoRA: Low-rank adaptation of large language models. CoRR abs/2106.09685 (2021) 14. Lafian, A.M., Torralba, K.D.: Lumbar spinal stenosis in older adults. Rheum Dis Clin North Am 44(3), 501–512 (2018) 15. Lin, C.Y.: ROUGE: A package for automatic evaluation of summaries. In: Text Summarization Branches Out: Proceedings of the ACL-04 Workshop. p. 74–81 (2004) 16. Nguyen, H.S., Vu, D.N., Nguyen, T.N., Van, B.T., Pham, V.D., Xuan, T.M., Vu, H., Luong, T.V.: CrossSpine: Multi-scale cross-sequence attention with anatomical priors for automated Pfirrmann grading. arXiv:2607.22728 (2026) 17. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. p. 311–318 (2002) 18. Salem, S., Habib, A., Guan, C.: AutoSpineAI: Lightweight multimodal CAD frame- work for lumbar spine MRI assessments. In: 2025 IEEE-EMBS International Con- ference on Biomedical and Health Informatics (BHI) (2025) 19. Sellergren, A., Kazemzadeh, S., Jaroensri, T., Kiraly, A., Traverse, M., Kohlberger, T., Xu, S., Jamil, F., Hughes, C., Lau, C., et al.: MedGemma technical report. arXiv preprint arXiv:2507.05201 (2025) 20. Vu, D.N., Nguyen, H.S., Nguyen, T.N., Van, B.T., Xuan, T.M., Vu, H., Lu- ong, T.V.: PhenSPINE: A standardized benchmark for spine pathology diagnosis. arXiv:2607.19696 (2026) 21. Wu, C., Zhang, X., Zhang, Y., Hui, H., Wang, Y., Xie, W.: Towards generalist foun- dation model for radiology by leveraging web-scale 2D&3D medical data. Nature Communications 16, 7866 (2025) 22. Xin, Y., Ates, G.C., Gong, K., Shao, W.: Med3DVLM: An efficient vision-language model for 3D medical image analysis. IEEE Journal of Biomedical and Health Informatics 30(3), 2524–2536 (2026) 23. Zanardo, M., Stoppa, T., Pegorer, S., Scapicchio, C., Chiappino, D., Di Vito, S., Bazzocchi, A.: Can AI write lumbar spine MRI reports like a radiologist? A blinded comparative study between LLM-generated and radiologist-written reports. Euro- pean Radiology Experimental 10(1), 16 (2026) 24. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: Evalu- ating text generation with BERT. In: International Conference on Learning Rep- resentations (ICLR) (2020) 25. Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., Van Gool, L.: CDDFuse: Correlation-driven dual-branch feature decomposition for multi- modality image fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 5906–5916 (2023)

Instant research discovery