Paper deep dive

Foundation Models for Medical Imaging: Status, Challenges, and Directions

Chuang Niu, Pengwei Wu, Bruno De Man, Ge Wang

Year: 2026Venue: arXiv preprintArea: eess.IVType: PreprintEmbeddings: 141

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%

Last extracted: 7/21/2026, 4:10:54 AM

Summary

This review paper analyzes the transformation of medical imaging from task-specific networks to large-scale Foundation Models (FMs). It covers principles of FM design (architectures like Transformers, CNNs, SSMs; training paradigms like generative and discriminative), applications across modalities (CT, MRI, etc.), and challenges regarding scalability, fairness, and clinical translation.

Entities (25)

Foundation Models · concept · 98%Bruno De Man · person · 95%Chuang Niu · person · 95%Ge Wang · person · 95%Medical Imaging · domain · 95%Pengwei Wu · person · 95%Transformers · model-architecture · 92%Convolutional Neural Networks · model-architecture · 90%CT · modality · 90%Diffusion Models · model-architecture · 90%GE HealthCare · organization · 90%MRI · modality · 90%Rensselaer Polytechnic Institute · organization · 90%Vision Transformers · model-architecture · 90%Swin Transformer · model-architecture · 88%Generative Adversarial Networks · model-architecture · 85%Image Reconstruction · task · 85%Mamba · model-architecture · 85%ResNet · model-architecture · 85%Segmentation · task · 85%Self-Supervised Learning · training-method · 85%State Space Models · model-architecture · 85%Transfer Learning · training-method · 85%U-Net · model-architecture · 85%Variational Autoencoders · model-architecture · 85%

Relation Signals (14)

Chuang Niu → affiliatedwith → Rensselaer Polytechnic Institute

confidence 95% · C. Niu, and G. Wang are with... Rensselaer Polytechnic Institute

Bruno De Man → affiliatedwith → GE HealthCare

confidence 95% · P. Wu and B. De Man are with GE HealthCare Technology & Innovation Center

Pengwei Wu → affiliatedwith → GE HealthCare

confidence 95% · P. Wu and B. De Man are with GE HealthCare Technology & Innovation Center

Ge Wang → affiliatedwith → Rensselaer Polytechnic Institute

confidence 95% · C. Niu, and G. Wang are with... Rensselaer Polytechnic Institute

Foundation Models → appliesto → Medical Imaging

confidence 95% · Foundation models (FMs) are rapidly reshaping medical imaging

Foundation Models → trainedon → Medical Imaging

confidence 90% · In medical imaging, FMs learn from large-scale, image-centric multimodal datasets

Foundation Models → usesarchitecture → Transformers

confidence 90% · Transformers have taken the lead in many domains, especially in NLP and image analysis.

Foundation Models → enablestask → Segmentation

confidence 85% · spanning 2D/3D segmentation

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Foundation models (FMs) are rapidly reshaping medical imaging, shifting the field from narrowly trained, task-specific networks toward large, general-purpose models that can be adapted across modalities, anatomies, and clinical tasks. In this review, we synthesize the emerging landscape of medical imaging FMs along three major axes: principles of FM design, applications of FMs, and forward-looking challenges and opportunities. Taken together, this review provides a technically grounded, clinically aware, and future-facing roadmap for developing FMs that are not only powerful and versatile but also trustworthy and ready for responsible translation into clinical practice.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

139,352 characters extracted from source content.

Expand or collapse full text

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 1 Foundation Models for Medical Imaging: Status, Challenges, and Directions Chuang Niu, Member, IEEE, Pengwei Wu, Member, IEEE, Bruno De Man, Fellow, IEEE, and Ge Wang, Fellow, IEEE Abstract— Foundation models (FMs) are rapidly reshaping medical imaging, shifting the field from narrowly trained, task-specific networks toward large, general- purpose models that can be adapted across modalities, anatomies, and clinical tasks. In this review, we synthesize the emerging landscape of medical imaging FMs along three major axes: principles of FM design, applications of FMs, and forward-looking challenges and opportunities. Taken together, this review provides a technically grounded, clinically aware, and future-facing roadmap for developing FMs that are not only powerful and versatile but also trustworthy and ready for responsible translation into clinical practice. Index Terms— Foundation models, medical imaging, image reconstruction, image analysis, multimodal learning I. INTRODUCTION rtificial intelligence (AI) for medical imaging is experiencing a transformative shift from task-specific models toward foundation models (FMs), which are large artificial neural networks pre-trained on vast, diverse datasets and adapted efficiently to a variety of downstream tasks. In medical imaging, where labels are scarce, heterogeneous, and expensive, FMs show a strong promise for rapid adaptation with minimal annotation, improved generalization across sites, scanners, and populations, and a plausible route to “generalist” medical imaging assistants that reason across contexts. Recent overviews from both the radiology and computer vision communities document a surge of FM research, spanning 2D/3D segmentation, image–text representation learning through vision–language fusion, and generative models. Together, these developments motivate a new synthesis of principles, capabilities, and translational considerations tailored to the healthcare ecosystem [1]. To contextualize foundation models, we begin by exploring their relationship with the broader AI landscape, coupled with Figure 1.1 illustrating the relative timelines of the related areas along with some seminal publications. AI refers to non-human systems performing tasks that mimic human perception and reasoning, such as language understanding and image analysis. Machine learning, a subset of AI, trains models to detect C. Niu and P. Wu are co-first authors with equal contributions, B. De Man and G. Wang are co-corresponding authors. C. Niu, and G. Wang are with Biomedical Imaging Center, Center for Computational Innovations, Center for Biotechnology & Interdisciplinary Studies, Department of Biomedical Engineering, School of Engineering, Rensselaer Polytechnic Institute, Troy, New York, USA (e-mail: niuc@rpi.edu; wangg6@rpi.edu). P. Wu and B. De Man are with GE HealthCare Technology & Innovation Center, Niskayuna, NY, USA (e-mail: pengwei.wu@gehealthcare.com; deman@gehealthcare.com). patterns in data, evolving from simple statistical methods to more sophisticated tools like random forests and support vector machines. Deep learning uses multi-layer artificial neural networks to represent data in a data-driven fashion, leading to advanced architectures, like Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Graph Neural Networks (GNNs), and Transformers. The term foundation model was coined by the Center for Research on Foundation Models at the Stanford Institute for Human-Centered Artificial Intelligence in August 2021 [19]. Foundation models are a class of deep learning models that are initially trained based on a diverse dataset for broad applications, and that can then be fine-tuned for specific down- stream applications. Typically, they are initially trained in a self-supervised fashion. These pre-trained FMs then serve as the basis for developing task-specific models through transfer learning. The term foundation model is sometimes used loosely: a critical examination of the criteria for a model to qualify as a foundation model is given in [20]. Foundation models are characterized by enormous training data and parameter counts, which lead to emergent capabilities that do not present in smaller models. In other words, a foundation model serves as a general-purpose platform that, with minimal task-specific training, can achieve strong performance across a variety of tasks. Another hallmark of foundation models is scalability. Their performance improves predictably as model size, training data, and amount of compute increase, following empirical scaling laws. This scaling yields surprising capabilities, e.g., GPT-3 demonstrated in-context learning to solve tasks it was not explicitly trained for. Foundation models also exhibit strong generalization and transferability, meaning that the knowledge captured during pretraining on broad data can be transferred to unseen tasks. A single pretrained model can be fine-tuned to excel in applications ranging from natural language processing (NLP) to computer vision and robotics. This versatility has incentivized homogenization of AI research around a few architectures, especially the Transformer. However, this also means any defects or biases in a foundation model might propagate to its downstream uses. We first introduce several previous review papers related to foundation models. A comprehensive survey of self-supervised A IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 2 Turing [2], Deep Blue[3], MYCIN [4], SHRDLU [5] IBM [6], Linear Regression [7], SVMs [8], Random Forests [9] LeCun [10], AlexNet [11], ResNet [12], LSTM [13], GAN [14], U-Net [15] DeepMind AlphaFold [16], Google Med-PaLM [17], Bommasani [18]. Fig. 1.1: Publication trends from 2000 to 2025 (Source: Scopus, Sep’ 2025) highlight the relative growth of Artificial Intelligence (AI), Machine Learning (ML), Deep Learning (DL), and Foundation Models (FM), alongside key milestones that shaped each field. learning (SSL) is provided by Liu et al in 2021 [21]. Two 2023 surveys by Kazerooni et al. and by Yang et al [22], [23] provide in-depth overviews of the rapidly evolving field of diffusion models, which are increasingly being integrated into foundation models. A more recent review of generative models is provided by Hein et al [24]. Longpre et al. [25] present a practical guide to support responsible and transparent development of FMs across text, vision, and speech modalities. Large language models (LLMs) are the most popular type of FMs. Zhou et al. [26] trace the evolution from BERT to ChatGPT, emphasizing key advancements in architecture, training methods, and model capabilities. Zhao et al. [27] summarize LLMs and emerging trends like multi-agent collaboration and chain-of-thought reasoning. Ian A. Scott [28] introduces physicians to FMs and LLMs, explaining how they can perform diverse tasks across modalities (text, audio, images, video), with potential applications in medicine. Yang et al. [29] offer an overview of ChatGPT, BERT, and other LLMs, detailing their underlying architectures, training strategies, and broad applications. Truhn et al. [30] explore how LLMs and multimodal foundation models are transforming precision oncology. Some excellent reviews focused on medical imaging and image analysis. Azad et al. [31] explore how FMs are reshaping the field of medical imaging, including a structured taxonomy of FMs in medical imaging and clinical applications, challenges, and directions. The vision-language models (VLMs) are covered in several reviews, including Huang et al. [32], Ryu et al [33], and Sun et al [34]. They analyze how multimodal FMs are reshaping clinical AI by integrating visual data (e.g., X-rays, MRIs) with textual information (e.g., radiology reports, clinical notes). Zhang et al [35], Huix et al [36], and Veldhuizen et al [37] survey FMs for medical image analysis and outline unique challenges of applying them in radiology, pathology, and ophthalmology. Khan et al [38] present an analysis of subgroup fairness in medical imaging FMs, indicating that improved overall accuracy may come at the expense of reduced subgroup fairness. The latest milestone in this emerging field is the IEEE Transactions on Medical Imaging Special Issue on Advancements in Foundation Models for Medical Imaging (2025), which assembled 18 papers spanning segmentation, multimodal integration, architectural innovations, benchmarking, ethics, and generative synthesis. Collectively, these contributions underscore both the breadth and depth of current progress: from SAM-inspired segmentation frameworks and Mamba-based backbones to multimodal vision–language adaptations and large-scale echocardiography models; from topology-guided generative pathology models to benchmark and ethical analyses that foreground fairness, interpretability, and governance. The Special Issue illustrates not only rapid technical advances but also the broader community recognition that foundation models for medical imaging and beyond must be judged by accuracy, equity, transparency, and clinical utility. This collection thus provides a valuable snapshot of the state of the art, while also motivating the need for integrative up-to-date reviews like the present article. This review advances the current literature in three distinct ways. First, it adopts a broad coverage of FMs in medical imaging, especially incorporating the underrepresented domain of image reconstruction for CT, SPECT, PET, MRI, ultrasound, and optical imaging. Second, by integrating the most recent developments in this rapidly evolving field, our review addresses temporal gaps in prior surveys, such as generative AI, reinforcement learning, and modern reasoning methods for medical imaging researchers and practitioners. Finally, we offer an extensive perspective as the last part that reflects our current vision to promote further advancement. The remainder of this review is organized as follows. In the next section, we distill the principles behind FMs that are most relevant to imaging, ranging from major model architectures, common training strategies, to the key components of development and deployment of FMs. In the third section, we survey applications across medical imaging modalities (CT, MR, PET, US, X-ray, ophthalmology, pathology) and tasks (segmentation, detection, diagnosis, triage, report generation, reconstruction), highlighting strengths, caveats, and challenges. In the final section, we identify future directions in terms of four pillars supporting medical imaging FMs, which are data/knowledge, model/optimization, computing power, and regulatory science. Overall, we hope to provide a unifying view that is technically grounded, clinically actionable, and forward- looking. IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 3 Fig. 2.1. Principles of Foundation Models. Overview of medical foundation models, illustrating how large-scale heterogeneous clinical data, including medical images with associated reports, lab tests, genetics, and electronic health records, are used to train scalable, generalizable, and transferable foundation models that can be adapted to diverse downstream applications such as radiology, oncology, pathology, neurology, and cardiology. The lower panel summarizes the major technical components, including model architectures, modeling paradigms and training workflows, and efficiency techniques for optimization and deployment. I. PRINCIPLES OF FOUNDATION MODELS In medical imaging, FMs learn from large-scale, image- centric multimodal datasets along with associated radiology reports, laboratory results, genetic profiles, and electronic health record (EHR) data. These models support a wide range of clinical specialties, including radiology, oncology, pathology, neurology, cardiology, and so on. This section outlines the core principles behind medical imaging FMs, covering model architectures, training strategies, and efficiency techniques, as summarized in Figure 2.1. A. Model Architectures Several neural network architectures serve as the building blocks for foundation models. Briefly speaking, Transformers have taken the lead in many domains, especially in NLP and image analysis. CNNs, however, often outperform Transformers on smaller-scale vision tasks when data is scarce, due to their built-in locality bias. For some tasks with very long sequential data, state space models (SSMs) like Mamba now show promising results, even surpassing Transformers of similar or larger sizes. In this subsection, we review the major architectures, their variants, highlighting strengths and limitations. 1) Transformer The Transformer [39] has become the de facto core of most FMs in language and increasingly in vision and medical imaging. Transformers dispense with recurrence and convolutions in favor of self-attention mechanisms that explicitly model long-range dependencies. A Transformer block is typically composed of multi-head self-attention layers and feed-forward layers, enabling it to attend to all positions of an input sequence in parallel. Vision Transformers (ViTs) [40], [41] tokenize images and analyze them based on sufficient training data. ViTs can achieve excellent performance across various vision tasks. ViTs’ strengths lie in their ability to capture global context easily via self-attention and their scalability with gradually refined attention coverage. However, since Transformers lack inductive biases, a ViT trained from scratch on limited data may underperform a CNN [41]. The Swin Transformer [42] was designed to address the ViT’s issues by computing self- attention in non-overlapping windows and shifting the window positions between layers to allow cross-window connections to reduce the computation and improve generalization on smaller datasets. Another strategy of reducing the computation for high- resolution/dimension images is to interleave global and local window attention across layers [43]. Decoder-only Transformers [44] are simplified Transformers where only the decoder stack is used. Through causally masked self-attention, each token can attend only to preceding tokens, making them inherently autoregressive. Importantly, causal self-attention is powerful in deployment due to the ability to perform key-value caching, which brings several tangible advantages, such as huge speedup in inference and stable latency. Thus, it has become foundational for modern language models. The decoder only architecture is also critical in multimodal modeling by decoding vision and language tokens for various tasks, such as image/video captioning [45] and medical report generation [46]. The Mixture of Experts (MoE) architecture [47] is an enhancement to Transformer architectures and replaces the ... ... Training Application •Scalability •Generalizability •Transferability Medical Images Reports Lab Test GeneticEHR RadiologyOncologyPathology Neurology Cardiology Large-scale DataDiverse Applications Generative Foundation Models ArchitecturesTrainingEfficiency Diffusion GANVAE AR Discriminative SSVRL CLIP Masked JEPA ViTDecoder-only MoE ResNet CNN-Attention Mamba H3 RWKV Quantization Efficient Attention Distillation FSDP / ZeRO Mixed PrecisionCheckpointing FlashAttention RetNet Pre-Training Generative Discriminative Training Workflow Modeling Paradigms Transformer Convolutional Models State Space Models Efficient Optimization Efficient Deployment UNet Kernel FusionGradient Accumulation vLLM/ SGLang/ Ollama RL PPO GRPO DPO Post-Training SFTRL LoRA 4 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 standard feed forward layer in each Transformer block with a set of parallel sub networks called experts and a learnable gating network that routes each input token to a small subset of experts based on token-specific features, dramatically increasing model capacity by only activating a fraction of the total parameters per token. Each gate typically employs a sparsely gated softmax function along with auxiliary load balancing losses and bias terms to distribute tokens evenly across experts [48]. This architecture enables training models with trillions of parameters, where only a few billion of those parameters are used during inference for any given token, making MoE an enabler of scalability in modern FMs. Efficient attention mechanisms address the quadratic cost of standard Transformers by reducing computation and memory while preserving essential context. Sparse attention limits each token’s receptive field through structured or adaptive sparsity [49]. Linear attention replaces softmax with kernelized or alternative formulations so attention scales linearly in sequence length [50], [51]. Low-rank and factorized approaches compress keys, values, or the attention matrix itself [52]. Meanwhile, multi-query and group-query attention [53], [54] share key–value projections across heads or head groups, boosting memory efficiency and throughput with minimal loss in expressiveness. Together, these methods form a toolkit that enables scalable Transformers for long-context, high- resolution, and multimodal tasks. 2) Convolution-based Models Convolutional Neural Networks (CNNs) [55] dominated computer vision for years, and they remain highly relevant in the era of FMs. CNNs like ResNet [56] and UNet [15] are good at learning local patterns and are translation-invariant. These inductive biases allow CNNs to generalize well with relatively small training datasets and excel on medium-scale tasks with strong local features. However, CNNs have a restricted receptive field, so localized convolutions might miss global context. Attention-Convolutional Models [57] aim to get the best of both CNNs and Transformers, such as by augmenting CNN backbones with attention blocks [58], or by augmenting Transformers with convolutional token embeddings [59]. However, they could also inherit some limitations of both paradigms. Nonetheless, these hybrid models form an important class of FM architecture. 3) State-Space Models Recurrent Neural Networks (RNNs) [60] were the workhorse for sequence modeling. However, RNN and its variants like Long Short-Term Memory (LSTMs) and Gated Recurrent Unit (GRUs) [61] faced challenges in capturing long-range dependencies. In particular, they could not be parallelized across sequence positions. To this end, State space models (SSMs) provide a powerful framework for sequence modeling by representing how hidden states evolve over time in response to inputs, offering a fundamentally recurrent alternative to attention-based architectures. Instead of computing pairwise interactions across all tokens, SSMs propagate information through a structured state update that can be computed efficiently in linear time, making them well-suited for very long sequences and streaming scenarios. The modern resurgence of SSMs began with the Structured State Space sequence model (S4) [62], which introduced stable diagonal-plus-low-rank parameterizations enabling long-range memory and efficient convolutional implementations. This foundation has since driven the development of highly expressive, scalable architectures such as Selective SSMs (Mamba) [63], RWKV [64], H3 [65], and RetNet [66], which have evolved into competitive sequence learners capable of matching or surpassing transformer performance in long-context tasks while offering significant advantages in scalability and memory efficiency. SSMs have been successfully adopted in medical imaging [67], [68]. B. Modeling and Training 1) Modeling Paradigms We can divide the various modeling methods for FMs into generative and discriminative/ contrastive paradigms. Generative models provide a full understanding of data and produce new examples, whereas discriminative/contrastive models excel at producing generalizable representations and making decisions. Generative models offer tools for data generation, uncertainty quantification, and discovering underlying data structure, which can be invaluable in improving medical imaging qualities. Contrastive, as a type of discriminative representation learning approach, currently dominate the pretraining for image analysis tasks such as classification, segmentation, detection, and regression, underlining predictive accuracy. Generative Modeling Variational Autoencoders (VAE) [69] exemplifies an early latent-variable generative approach that marries probabilistic models with deep learning. A VAE consists of an encoder network that maps input data to a latent distribution and a decoder network that reconstructs the data from a latent sample, trained jointly by maximizing a variational lower bound on data likelihood. This framework enables learning a deep latent representation while permitting effective and efficient inference. VAEs have been pivotal as a principled method for learning unsupervised generative models of images, offering stable training and explicit probability density estimation. Furthermore, some extensions of VAEs were proposed; e.g. β- VAE [70] for disentangled factors, VQ-VAE [71] for discrete latent spaces. VAEs generate new samples rapidly but often at a relatively compromised quality in comparison with more recent models introduced below. In generative foundation models, VAEs serves as an important method for vision tokenization/compression [72]. Generative Adversarial Networks (GANs) [73] are a major step forward for generative modeling, formulated as a minimax game between a generator that synthesizes data and a discriminator that distinguishes real from fake data. Successive innovations such as DCGAN [74], Progressive GAN [75], StyleGAN [76], and BigGAN [77] improved training stability, scale, and controllability. In medical imaging, GANs have been widely adopted for cross-modality translation [78], super- 5 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 resolution [79], denoising [80], inpainting [81], and simulation [82], serving as both a critical modeling paradigm in the evolution of FMs and a practical tool in medical imaging. While GANs remain attractive for their efficiency at inference, they often suffer from training difficulties and mode collapses. Diffusion models [83] are a recent class of generative models that achieve state-of-the-art results. These models define a forward process that gradually adds noise to every image or sample in a training dataset until it becomes pure noise. Then, a learned reverse process will remove noise gradually to synthesize a new image or sample. The seminal Denoising Diffusion Probabilistic Model (DDPM) [83] demonstrated that diffusion models can produce excellent images, typically outperforming GAN results while offering advantages like stable training and distribution coverage. A main drawback of DDPM is the computational cost: generating an image requires many iterative denoising steps, making them slower than one- shot generators like VAEs and GANs. Recent research addresses this drawback with optimized samplers which allow fewer or even just one step, such as latent diffusion techniques [84] which diffuse in a lower-dimensional latent space to speed up generation and consistency models [85] which remove noise in one or few steps. Diffusion models have rapidly been adopted in medical imaging for various tasks, such as image reconstruction and enhancement [86]. These diffusion models are based on thermodynamics and have been extended in reference to electrodynamics and other mechanisms [86]. Autoregressive (AR) [87] generative models treat data synthesis as a sequential prediction problem, modeling the joint distribution of high-dimensional data as a product of conditionals. In natural language processing, this framework underlies the next-token prediction mechanism in LLMs, where each token is generated by conditioning on all previously generated tokens [44]. In computer vision, early examples include PixelRNN [88] and PixelCNN [89], which demonstrated that images can be generated pixel-by-pixel by scanning an image field and predicting the next pixel intensity conditioned on the context. However, they are notoriously slow since all output elements are produced sequentially. Recent advances demonstrate that images can be first compressed into discrete latent codes and then modeled with a Transformer as an autoregressive sequence of tokens, as illustrated by DALL·E [90] for text-to-image generation. This paradigm has proven successful in natural language processing first and more recently for image generation [91], and even in multi-modal tasks [92]. Discriminative Modeling Self-supervised visual representation learning (SSVRL) exploits large-scale unlabeled images to learn features. Discriminative self-supervised methods are essential for SSVRL. These methods do not attempt to model the input distribution fully; instead, they train neural networks on pretext tasks such that solving these tasks requires extracting high-level semantic features. One prominent class of methods is contrastive learning, exemplified by methods like CPC [93], SimCLR [94], MoCo [95], and PIRL [96]. Furthermore, the teacher-student learning paradigm, such as in BYOL [97], SimSiam [98], and DINO [99], took a surprising step by removing explicit negative pairs in contrastive learning. Despite the absence of contrasting against negatives, these methods avoid collapse through their asymmetric teacher- versus-student architecture. Clustering-based methods [100], [101], [102] leverage self-supervised visual representation learning, achieving excellent results. Recently, information maximization methods emerged as a promising direction for self-supervised learning due to their simplicity without contrastive negative examples nor asymmetric design [103], [104]. Self-supervised discriminative learning enables rich feature learning from unlabeled datasets, greatly reducing the need for costly annotations in medical imaging [105], [106]. Vision-Language Contrastive Learning learns joint representations from paired images and text. These approaches, such as CLIP [107] and ALIGN [108], extend contrastive learning to multimodal data: an image and its accompanying caption form a positive pair, and mismatched image–caption combinations form negatives. By leveraging extremely large datasets of image–text pairs, these models learn remarkably general and transferable visual features. After CLIP-based multimodal pretraining, the image encoder may use zero-shot training for classification. By training on noisy but abundant web data, these models encode a rich association between visual concepts and natural language, enabling important downstream applications [107]. In the medical imaging field, analogous approaches have been extensively explored, e.g. aligning radiology images with report text, to bring the benefits of multimodal pretraining to specialized domains [109]. Generative-Discriminative Modeling Masked autoencoders reconstruct missing or corrupted portions of the input. A prime example in NLP is BERT [110], which learns a deep bidirectional Transformer by masking out random words in a sentence and training the model to predict missing tokens. In computer vision, the Masked Autoencoder (MAE) [111] blocks a fraction of image patches and then reconstruct them, even outperforming supervised pre-training on the same architecture for downstream tasks. The MAEs are optimized via reconstruction error rather than likelihood. As such, they straddle generative and discriminative paradigms: the training objective is generative, but the resulting encoder is typically used for discriminative tasks. Joint-Embedding Predictive Architecture (JEPA) [112] builds on the idea of a “world model”, learning by predicting future or missing higher-level representations. A recent instantiation is I-JEPA [113].Another example is data2vec [114]. These approaches define an appealing middle ground that combines embedding, alignment, and prediction, learning both generative and discriminative features across modalities to support intelligent behavior. This paradigm resonates strongly with the Bayesian brain hypothesis [115] and the minimum free-energy principle [116], which similarly view intelligence as predictive modeling of latent structure in the world. Reinforcement Learning Reinforcement learning (RL) has become critical in training 6 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 FMs, providing a principled mechanism to optimize model performance beyond supervised learning. While large-scale pretraining equips models with broad linguistic and world knowledge, RL enables them to incorporate explicit evaluative signals—ranging from human feedback to verifiable task-based rewards—into their policy updates, thus facilitating goal- directed refinement of model capabilities. This transition from next-token prediction to preference-aligned optimization reflects a most important trend in foundation model research, where performance reliability, controllability, generalization, and interpretability increasingly depend on an iterative feedback loop [117], [118]. RL-based methods help ensure that model outputs adhere to human-preferred behaviors, safety norms, and interaction standards. The canonical RLHF pipeline centers on Proximal Policy Optimization (PPO) [119], paired with learned reward models derived from human preference data [120]. This KL- regularized objective enables stable policy updates while preventing divergence from a reference policy. More recently, Direct Preference Optimization (DPO) [121] has emerged as a compelling alternative, reformulating the KL-constrained preference-alignment objective into a tractable supervised- learning–style loss that eliminates the need for value estimation and on-policy sampling. Complementary formulations, such as ORPO [122], further streamline preference optimization by merging likelihood training with preference modeling. In medical imaging, these methods were applied to radiology report generation [123] and radiology question answering systems [124]. For reasoning-intensive domains, models are optimized for verifiable correctness and multi-step reasoning quality. This shift has catalyzed the adoption of Group Relative Policy Optimization (GRPO) [125] and its variants. These methods compute relative advantages among multiple sampled trajectories for each prompt, thereby avoiding explicit critics and improving stability in domains where correctness signals are sparse but reliable. Enhancements such as DAPO [126] and GSPO [127] further refine group-based policy-gradient dynamics for large-scale reasoning optimization. Beyond GRPO, a growing literature explores RL with verifiable rewards [128], tree-search–augmented RL [129], self-play– driven reasoning improvement [130], and offline RL for complex reasoning trajectories [131]. Collectively, these methods position RL as a cornerstone for advancing FMs from superficially coherent reasoning to demonstrably correct, logically structured problem solving. RL-based reasoning methods have been explored for medical imaging applications, such as medical image question answering [132] and personalized lung cancer risk prediction [133]. C. Training Workflow A typical training workflow of FMs involves a large-scale pre-training stage for learning generalizable representations, followed by an iterative post-training process including supervised fine-tuning (SFT) and reinforcement learning for alignment and/or reasoning. Although different types of FMs may adopt varied strategies, this two-stage paradigm remains the dominant developmental framework. Pre-training on Broad Data: The pre-training stage typically leverages large-scale, heterogeneous datasets to learn robust representations and capture complex data distributions. For text and vision transformers, this involves billions of text tokens or millions of images, optimized through self-supervised objectives such as masked language modeling (BERT), next- token prediction (GPT), or contrastive alignment (CLIP). In biomedical domains, “broad data’ further includes large collections of clinical text, medical images, and multimodal corpora, enabling the development of domain-specialized models such as BioLMs [134], MedSAM [135], and M3FM [136]. In many cases, domain-adaptive pretraining is performed beforehand where models are further pretrained on large in-domain corpora (e.g., BioBERT [134] trained on PubMed) to improve handling of domain-specific terminology and semantics. The pre-training of generative models such a s diffusion models [84] and GANs [137] learns to approximate the underlying data distribution closely enough to generate realistic samples or reconstruct missing or corrupted information. This generative pre-training enables models to internalize fine-grained structural and semantic patterns, while providing additional benefits unique to synthesis-based objectives. In medical imaging, such pre-trained generative models have been successfully applied to medical image reconstruction [138], super-resolution [139], denoising [140], and robust artifact correction [141]. Despite its computational complexity, pretraining is essential to extract implicit, rich, and transferable features and knowledge for downstream adaptation [136]. Supervised Fine-Tuning: After large-scale pre-training, models are specialized for target domains or downstream tasks through fine-tuning on smaller, high-quality labeled datasets, typically with reduced learning rates to preserve general features. A central challenge is balancing specialization with the preservation of general knowledge. Risks of overfitting and catastrophic forgetting are addressed through several effective techniques, such as layer freezing, adaptive optimization, and parameter-efficient adaptation methods (e.g., LoRA [142]). Reinforcement Learning for Alignment and Reasoning: A critical aspect in the development of FMs is alignment, which ensures that outputs are not only plausible and fluent but also accurate, reliable, and consistent with domain standards and human values. A widely adopted paradigm is Reinforcement Learning from Human Feedback (RLHF), wherein human evaluators provide preference rankings of model outputs that are distilled into a reward model, subsequently optimized via reinforcement learning algorithms [119]. More recently, reinforcement learning has been renovated to improve reasoning quality. Reasoning-oriented models such as DeepSeek-R1[143] embody this shift of reinforcement learning [125] to encourage multi-step problem solving, self- consistency, and robustness in complex decision-making tasks. This shift underscores the versatility of reinforcement learning both as an alignment mechanism and as a route to strengthen reasoning performance, an essential property for high-stakes 7 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 medical imaging applications. In some training pipelines, data generated after RL is fed back into the SFT dataset, creating an iterative loop that progressively improves real-world performance [143]. D. Computational Efficiency The rapid scaling of FMs has intensified the need for computationally efficient training and cost-effective deployment, especially for medical imaging applications where the data are of high dimensions. Efficiency is now a foundational design goal rather than a secondary consideration. Although some architectural efficiency designs such as MoE and efficient attention mechanisms have been introduced, this section reviews critical and popular techniques used to optimize training and inference phases of large-scale models. 1) Efficient Optimization Training FMs is computationally challenging. Efficient optimization techniques address this challenge by reducing memory overhead, improving parallelism, and leveraging hardware capabilities, with some important techniques reviewed below. Sharded Training and Memory Partitioning: Fully Sharded Data Parallelism (FSDP) [144] and ZeRO-based [145] optimization represent two state-of-the-art strategies for memory-efficient distributed training. These methods partition model parameters, gradients, and optimizer states across devices, enabling training of models that exceed the memory limits of individual GPUs. The ZeRO family (ZeRO-1/2/3) further allows flexible control over the trade-off between memory savings and inter-device communication. Parameter-Efficient Fine-Tuning: Low-Rank Adaptation (LoRA) [146] and its variants [147] reduce the computational and storage cost of fine-tuning by introducing lightweight low- rank matrices into Transformer modules. Only these small sets of parameters are updated during training, allowing adaptation to new tasks with orders-of-magnitude fewer trainable weights. Efficient Attention and Kernel Implementations: FlashAttention [148] provides an optimized, memory-efficient attention implementation that minimizes redundant data movement, directly improving training speed. Similarly, kernel fusion techniques [149] combine multiple GPU operations into a single execution step, reducing kernel-launch overhead and improving hardware utilization. Mixed Precision and Activation Checkpointing: Mixed- precision training [150] with FP16 or BF16 formats reduces memory footprint and increases arithmetic throughput, particularly on modern Tensor Core devices. Activation checkpointing [151] further reduces memory use by storing only a subset of intermediate activations and recomputing others during backpropagation. Together these techniques allow larger batch sizes and deeper models to be trained with the same hardware resources. Gradient Accumulation: When batch sizes exceed GPU memory capacity, gradient accumulation simulates large-batch training by aggregating gradients across several forward passes [152]. This enables stable optimization behavior without requiring large accelerator clusters. 2) Efficient Deployment Once training is complete, model deployment must balance performance, latency requirements, and hardware constraints. Efficient deployment focuses on compressing models, reducing precision, and leveraging specialized inference engines to achieve high-throughput, low-cost inference. Model Compression via Distillation: Knowledge distillation [153] transfers the behavior of a large “teacher” model to a smaller “student” model. The resulting student model retains much of the teacher’s predictive capability while requiring substantially fewer parameters and reduced computing. Distillation is especially effective for edge devices, mobile platforms, and latency-sensitive applications. Quantization for Reduced Precision Inference: Quantization converts model weights and activations from high-precision formats (e.g., FP32) to lower-precision representations such as INT8 or INT4 [154]. Modern quantization-aware and post-training quantization techniques maintain accuracy while dramatically improving inference throughput and lowering memory usage. These methods have become standard in production-scale model serving. Specialized Inference Runtimes: Recent advancements in inference system design have led to optimized runtimes tailored specifically for LLMs and multimodal LLMs. • vLLM [155] introduces PagedAttention, an efficient memory management mechanism that improves multi-request batching and maximizes GPU utilization. • SGLang [156] extends these ideas by incorporating structured caching and partial-decoding reuse, enabling high- throughput, multi-tenant deployment scenarios. • Ollama focuses on ease of deployment for local environments, particularly for quantized models running on consumer hardware. These systems provide substantial improvements over traditional deep learning servers, achieving significantly higher throughput for practical applications. I. MEDICAL IMAGING APPLICATIONS In this section, we survey successes and challenges of FMs in medical imaging for various modalities and tasks as shown in Fig. 3.1, and we list the major benchmarks and medical imaging data platforms for development of FMs. Medical imaging methods often face the long-tail data scenario, caused by heavily imbalanced datasets in which many common disease cases coexist with fewer rare disease cases. Consequently, the scarcity of data for training models to accurately identify these rare cases leads to performance degradation. The few-shot setting of FMs aligns perfectly with this long-tailed scenario, serving as a versatile base for a wide range of imaging modalities, anatomies, and downstream tasks. A. Image Reconstruction and Enhancement Medical image reconstruction involves solving inverse problems to recover high-quality images from incomplete or corrupted data. For example, reconstructing images from undersampled k-space signals or noisy/incomplete sinograms 8 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 [157], [158], [159], [160]. FMs are increasingly researched for this purpose to improve reconstruction quality under challenging conditions for various imaging modalities. Instead of training a separate network for every scanner or protocol, a single large model can be pretrained on diverse image data and then adapted to specific reconstruction tasks. This section focuses on three important types of foundations models for image reconstruction & enhancement tasks as shown in Fig. 3.2. Fig. 3.1. Example applications of foundation models in different aspects of medical imaging, across different modalities, anatomies, and tasks. 1) Image Enhancement These types of networks directly improve the data quality, either in a measurement or image domain, or in both domains as shown in pink in Fig. 3.2. Common examples include sinogram completion and inpainting, K-space completion, denoising [80], [161], super-resolution [79], artifacts correction, inhomogeneity correction, and harmonization networks [162]. They do not require explicitly defined forward models, since they are not solving an inverse problem. Early attempts have demonstrated that deep learning can greatly enhance image quality in specific scenarios. However, these models typically are limited to pre-defined tasks or selected anatomic regions, and they often suffer from poor generalizability for out-of-domain tasks. Recently, there has been an increasing interest in FMs for multiple image enhancement tasks. TAMP for example leverages a physics- drive pre-training and parameter-efficient adaptation for universal CT image quality improvement in both sinogram and image domains [163]. 2) Direct Reconstruction Researchers began leveraging large, paired datasets to directly learn mappings from measurements (e.g., sinograms in CT, k-space in MRI) to the final clean, high-quality images in an end-to-end fashion [164]. These approaches focus on minimizing reconstruction and other losses (e.g., L1, L2, perceptual [165], adversarial [165]) during the training procedure. They are typically not iterative in nature and can use imaging system models to enforce physical constraints. Compared to image enhancement networks, reconstruction networks can incorporate the measurement operator and observed data. For example, the “Reconstruct Anything” model proposes a universal direct inversion model by introducing a new conditioning mechanism that integrates the imaging physics through multigrid Krylov iterations [166]. This single backbone model performs multiple image reconstruction and enhancement tasks. Another prominent example is the unrolled network framework type of approach, which mimics iterative optimization algorithms (e.g., ISTA, ADMM) by embedding data consistency and learned regularization into a trainable architecture [157], [158], [159], [160]. Finally, another line of work aims to directly learn the inverse model with deep learning (i.e., not explicitly providing the imaging physics / forward model to the network), for example, iRadonMap [167], [168], hierarchical DL reconstruction [169], and AUTOMAP [170], [171]. However, making the network learn an inverse models can be challenging due to their dimensionality. We distinguish these methods from image enhancement methods since they involve a domain change between input and output (e.g., from K-space to image domain). This is still an emerging application for foundation models. 3) Prior Modeling The final category of networks focuses on prior distribution modeling. Such a model can help the inversion/reconstruction process significantly. Instead of relying on paired data, they can naturally be applied to different inverse problems without fine- tuning. Most of these methods need the forward model to enforce various conditions such as tomographic data to finish tomographic reconstruction. While most of these methods are iterative, they are not the defining characteristics of the proposed method; for example, a conditional diffusion model trained to correct for different types of artifacts would still classify as an image enhancement task, despite being iterative in nature. One popular line of work utilizes score-based models to estimate the unconditional score function of the prior distribution. Then, during inference, some form of measurement matching technique is used to condition the reverse diffusion with a closed-form, approximated measurement matching score. Some examples are DPS, Score ALD, Score-SDE, Repaint (for inpainting only), BlindDPS, DDRM, PFGMs [24], DDS [172], Blaze3DM [173], and FORCE [174]. Another line of work uses plug-and-play networks [175], [176], which use a pre-trained denoiser to regularize an iterative reconstruction process of solving inverse problems, as opposed to using a hand-crafted regularizer such as total variation [177] or wavelet sparsity [178]. While all the above-mentioned methods can potentially be used for various applications (thus qualifying as FMs), the first two categories often require retraining for new applications. In contrast, the third category offers a general framework for arbitrary inverse problems without retraining nor fine-tuning, IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 9 Fig. 3.2. Illustration of foundation models for different tasks in medical imaging. The center portion shows three types of foundation models for image reconstruction & enhancement. Pink: image (or data) enhancement model. Blue: direct reconstruction model. Orange: prior distribution model. The other portions show data generation models (green) and data analysis models (yellow). likely at a cost of slow inference speed. Overall, while the use of FMs for tomographic reconstruction is still emerging, the trend is clear: a single large model (often generative) can flexibly handle multiple inverse problems by virtue of comprehensive prior knowledge. In major clinical areas, this means faster scans (by reconstructing from sparse data), lower radiation and contrast doses, higher image quality, and better diagnostic performance. B. Image Analysis In medical image analysis, innovative FM techniques are continually emerging in various clinical tasks. A few representative applications are summarized here. 1) Classification and Regression FMs have driven advances in medical image classification and regression, enabling diagnostic prediction and feature discovery with minimal supervision. Early deep learning models already matched expert performance in tasks like disease detection from images, but they required large, labeled datasets. FMs address this challenge by leveraging self- supervised pretraining on vast unlabeled datasets, often coupled with text, to reduce labeling efforts in various domains. In radiology, the CheXzero model [179] was trained on hundreds of thousands of chest X-rays and their clinical reports using contrastive vision-language learning. CheXzero achieves zero-shot pathology classification, i.e., it can detect diseases that were not annotated, reaching area-under-curve (AUC) values around 0.95 for several findings on external X-ray datasets. In oncology, FMs are used to discover imaging biomarkers from radiological scans. Pai et al. trained a self- supervised encoder on 11,467 diverse tumor images [180], yielding a model that outperformed conventional supervised methods in predicting clinical biomarkers, especially in low- data regimes. In pathology, FMs are also pivotal for classification like cancer subtyping. For example, Prov- GigaPath [181] achieved state-of-the-art accuracy on 25 of 26 tasks in a pathology benchmark (covering cancer subtypes and “pathomics” predictive tasks), significantly outperforming prior methods on the majority of those tasks. Finally, regression-based approaches remain central to cancer risk assessment and survival prediction, as indicated by studies such as DeepSurv [182] and TabSurv [183]. 2) Segmentation and Detection Early attempts at deep learning for segmentation has relied on task-specific models, each requiring laborious annotations [184]. FMs are redefining this area by learning generalized segmentation capabilities across diverse organs and modalities. Several studies explored how large-scale pretrained models can generalize across segmentation tasks in medical imaging. Ma et al. [185], and Noh et al [186] provide overviews of FMs for segmentation tasks, including tumor detection, organ delineation, cell segmentation, and anomaly identification. The Segment Anything Model (SAM) [187] is a promptable 10 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 paradigm and the first FM for general-purpose image segmentation. However, applying it naively showed limited accuracy on many medical images [188]. To bridge this gap, researchers developed medical-domain variants like MedSAM [189], a FM for “universal” medical image segmentation, which was pretrained on 1.57 million image–mask pairs spanning 10 imaging modalities and over 30 disease types. It was evaluated favorably on 86 internal tasks and 60 external test tasks. Other attempts include MedLSAM [190], 3DSAM-Adapter [191], SAM-Med2D [192], SAMed-2 [193], and SAM-U [194], enhancing the sensitivity and specificity of medical image segmentation tasks. However, promptable segmentation can be time-consuming in 3D cases for large cohort data analysis. Given that both SAM and SAM2 (video variant of SAM) handle either single or a stream or 2D images, their performance is typically worse than specially trained 3D models. To address this issue, various 3D medical segmentation models (with and without prompts) were proposed. Promptless models typically support automated segmentation of most human anatomies and major pathologies directly [195]. For example, BrainSegFounder uses SwinUNETR for 3D neuroimage segmentation tasks [196]. VISTA3D is a 3D segmentation model, whose performance was boosted by distilling state-of-the-art 2D image segmentation with supervoxels [196]. 3) Registration Conventional registration algorithms (e.g. deformable registration) are generic but slow [197], while recent learning- based methods are fast but tend to overfit [198], [199], [200], [201]. Emerging FMs deliver state-of-the-art results. UniGradICON[202], [203] is an early FM for medical image registration, trained on a dozen public datasets covering various anatomies. This model achieved high accuracy across multiple registration tasks (e.g., aligning brain MRI scans as well as thoracic CT scans). Similarly, Hu et al enhanced the robustness and generalizability of registration using a FM after sharpness- aware minimization [204]. Another line of work focuses on performing zero- or few- shot transfer learning using pretrained vision models. For example, DINO-REG uses the feature maps from DINO (a natural image foundation model) to compute the registration loss [205]. MultiCo3D leveraged anatomical information from the SAM model to guide registration (via aligning semantics) [206]. FoundationMorph utilizes a pretrained vision-language model to guide registration and a multi-dimensional attention module to fuse vision-language representations [205]. C. Image Generation Image generation especially relevant in medical imaging due to data scarcity including data imbalances (rarity of certain medical conditions and/or populations), high human-annotation cost, and patient privacy concerns [207]. By creating artificial yet realistic medical images with generative AI, one can greatly reduce the dependency on real patient data for training powerful deep learning models [208], [209], virtual clinical trials [210], [211], and training medical professionals [212]. FMs have revolutionized image generation by enabling scalable, high- fidelity synthesis across diverse modalities. Recent efforts demonstrated that generative AI can synthesize high-quality chest X-ray images, 3D MR and CT images, 2D pathology images, and so on [213], [214], [215], [216], [217], [218]. We highlight two research aspects of image generation: Model Architecture & Conditioning: Early attempts at medical image generation typically rely on generative adversarial networks (GANs), which – while powerful - suffered from issues like mode collapse, difficulty in training, and limited sample diversity [219]. Transformers have then been used (e.g., Med-Art [220], TransMed [221], MedFormer [222]) to offer global context modeling and scalability. Recently, diffusion-type models have emerged for high-quality image synthesis [213], [214], [215], [216], [217], [218]. For example, MINIM has presented a unified text-conditioned latent diffusion model for different domains, including OCT, chest X-ray, and CT [218]. In terms of new conditioning approaches (i.e., controlled generation): one line of work uses text-to-image diffusion models adapted to medicine. For example, RoentGen [223] fine-tunes a popular latent-space vision–language diffusion model (Stable Diffusion, trained on natural images and captions) using tens of thousands of chest X-rays paired with radiology report sentences. With DiffTumor [224], CT scans can be synthesized with a liver tumor of a specified size and location. Beyond text prompts, FMs can generate images conditioned on other inputs, such as segmentation maps or existing images (image-to-image translation). Diffusion models are also used to inpaint or modify medical images in a controlled way [225]. Scalable Generation: Realistic high-resolution 3D/4D volume generation is rather challenging due to the high memory footprint required by a unified 3D framework. This is further complicated by the inhomogeneities of medical images in terms of volume dimensions and pixel sizes. GenerateCT [226] addresses this challenge by decomposing the 3D generation process into a sequential generation of individual slices. While a volume can be generated to arbitrary sizes with this approach, there are lingering concerns regarding the 3D structural inconsistencies across slices. Others attempt at direct 3D image generation: for example, MAISI achieves 5123 realistic CT image generation via latent space diffusion and Tensor splitting parallelism (TSP) [227], [228]. D. Report Generation and Vision Question-Answering FMs have substantially advanced automated radiology report generation. For example, FMs can generate human-readable reports from multi-modal data in structured or unstructured formats in professional and plain languages [229], [230]. Compared with earlier encoder–decoder systems, modern multimodal Transformers pretrained on datasets such as MIMIC-CXR can produce more coherent, structured, and clinically aligned reports, often with explicit sections for findings, impressions, and comparisons [231]. Recent work further applies preference-based optimization such as Direct Preference Optimization (DPO) to suppress hallucinated prior examinations and better align generated text with radiologist IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 11 TABLE 1. MAJOR DATASETS AND PLATFORMS FOR MEDICAL IMAGING. Name / Platform Modality / Type Scope (Anatomy / Task) Annotation / Data Type Scale ChestX-ray14 [39] X-ray Chest diseases Image-level labels 100k+ CheXpert [40] X-ray Chest (in/outpatient) Uncertain labels 220k+ MIMIC-CXR [41] X-ray + reports Chest (ICU) Full radiology reports; labels 370k+ DeepLesion [42] CT Whole-body lesions Bounding boxes 30k+ slices RadImageNet [43] CT/MRI/US Multi-organ, multimodal Diagnostic labels 1.3M+ BraTS [44] MRI (3D) Brain tumors Segmentation masks ~2k cases EchoNet-Dynamic [45] Ultrasound (video) Cardiac EF values; masks 10k+ videos TCGA [46] WSI + CT/MRI/PET Multicancer Dx labels; genomics; ROIs 20k–30k WSIs + radiology PANDA [47] Pathology (WSI) Prostate Gleason grading 11k ROCO[48]/MedICaT [49] Multi-modality + text Scientific figures Captions; article text 80k–200k MedMNIST v2 [50] Multi-modality 18 organ/task datasets Class labels ~700k MSD [51] CT/MRI 10 organs Pixel masks 2.6k volumes fastMRI [52] MRI Knee, brain Raw k-space; fully & undersampled Millions of slices Calgary-Campinas [53] MRI Brain Fully sampled k-space ~370 volumes MRiLab synthetic MRI [54] MRI (simulated) Brain Synthetic k-space Millions Mayo LDCT Challenge [55] CT Chest/abdomen Full-dose/low-dose paired CT Hundreds of volumes LIDC-IDRI [56] CT Lung Nodule labels; full CT volumes 1,018 cases TCIA [57] Platform (multi-modality) Multicancer, multi-organ Curated datasets; Dx; segmentation; genomics 50k+ studies MIDRC [58] Platform (X-ray, CT) COVID-19, thoracic Standardized imaging + metadata 500k+ images UK Biobank Imaging [59] Platform (MRI, X-ray, fundus) Population cohort Structural/functional MRI; clinical data 100k+ participants NLST [60] CT, Clinical Data Lung cancer screening Nodule annotations; CT series Tens of thousands OASIS [61] / HCP [62] MRI Neuroimaging Structural + functional MRI Thousands AAPM Challenges [63] Multi-modality CXR, CT, colonoscopy, fractures Task-specific annotations Va r i e s PhysioNet Imaging [64] Platform (X-ray, US, CT) ICU and clinical cohorts Images linked to EHR/waveforms 100k+ expectations [232]. Vision question-answering (VQA) provides a novel interaction mode in which FMs answer targeted queries such as “Is there cardiomegaly?” or “What is the size of the liver lesion?” rather than producing a full report. Leveraging large- scale image–text pretraining and attention-based localization, contemporary medical VQA systems achieve strong performance across CXR, CT, MRI, ultrasound, and pathology, and can often operate in zero-shot or few-shot regimes [231]. Alignment methods like DPO have also been adapted to radiology VQA (RadQA-DPO) [233]. E. Other Tasks Finally, we describe how FMs may solve some applications peripherally related to medical imaging. FMs can automate patient follow-up. Using imaging data and electronic patient records, FMs can personalize messages with recommendations and appointments. FMs can identify public-health relevant disease patterns or biomarkers. For example, FMs were suggested for detecting quantitative cancer biomarkers and predicting public disease progression [234]. FMs can also be used for workflow optimization, such as image quality monitoring in a hospital, which can then alert technologists and/or physicians to image quality issues and/or incorrect protocol selection [235], [236], [237]. Other usages include automatic protocol selection and recommendation based on scout images and patient information, and automatic data management (e.g., similar case retrieval, [238]). This list seems endless, only limited by our imagination. F. Medical Imaging Datasets and Benchmarks Foundation models in medical imaging rely on large, diverse, and multimodal datasets that span radiography, CT, MRI, ultrasound, nuclear medicine, digital pathology, and image–text corpora, as well as highly specialized datasets for tasks like reconstruction, segmentation, and report generation. Public platforms such as The Cancer Imaging Archive (TCIA) [239] and the Medical Imaging and Data Resource Center (MIDRC) [240] have become central hubs for standardized, curated collections across cancer imaging, COVID-19 imaging, lung screening, and multi-organ cohorts, enabling reproducible benchmarks and large-scale pretraining. In parallel, dedicated CT and MRI reconstruction datasets—such as fastMRI [241], MRiLab data [242], the Mayo Clinic Low-Dose CT Challenge datasets [243], and AAPM Grand Challenge collections [244]—provide high-quality raw k-space or projection data needed to support physics-informed foundation models. Table 1 summarizes representative datasets and platforms that underpin the development and evaluation of generalist and multimodal foundation models in medical imaging. IV. PERSPECTIVES The successes of AI have been commonly attributed to the three pillars: data, models, and computing power [265]. Large, diverse, and multimodal data fuel the learning process; advanced architectures and optimization techniques extract information from data to empower generalizable models; and computing infrastructure enables the training and deployment of AI systems. Together, these three pillars have driven much of the rapid evolution of foundation models in medical imaging. However, medicine is a mission-critical domain, and medical imaging serves as the eyes of modern medicine, where errors IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 12 Fig. 4.1. Four pillars of foundation models for medical imaging. While three pillars for AI progress have been widely recognized, including data, models, and computing, in medical imaging the stakes demand the fourth pillar: regulatory science. have life-altering consequences. Unlike other fields where innovation could temporarily outpace regulation, healthcare demands not only technical sophistication but also trust, safety, and accountability. This reality calls for the fourth pillar: regulatory science. Robust evaluation frameworks, fairness auditing, clinical trials, and alignment with ethics ensure that foundation models for medical imaging meet the highest standards before entering real-world clinical workflows. By explicitly adding regulatory science to the traditional triad as shown in Figure 4.1, we underline that AI in medical imaging demands more than just bigger datasets, smarter models, and faster GPUs – it requires a disciplined pathway from technical breakthroughs to clinical integration. A. Data/Knowledge An unprecedented scale of datasets has been used in pretraining large AI models including foundation models. However, it is increasingly recognized that bigger is not always better, particularly in the medical domain where image subtleties, clinical representativeness, and biological complexities directly affect model outcomes [65]. While the size of a dataset is a good indicator of information content, it has become the consensus that data quality, diversity, and multimodality are equally important for development of trustworthy AI models and their clinical performance. Indeed, data quality ensures that foundation models learn meaningful patterns rather than noise or artifacts. Medical images vary widely in type, quality, style, and annotation accuracy. Low- quality or inconsistently labeled data can propagate errors through downstream tasks. Also, data diversity underpins generalizability across patient populations, scanners, and clinical settings. Many existing datasets overrepresent specific demographics or disease types. Finally, integrating imaging with clinical text, genomics, and longitudinal health records, multimodality unlocks representations that reflect the full complexity of patient care. Despite their value, datasets remain largely fragmented across hospitals, vendors, and jurisdictions due to privacy regulations (e.g., HIPAA, GDPR), intellectual property concerns, and institutional policies. Federated learning has already served as an alternative [66]. To further improve privacy, secure computation techniques can be combined with federated learning to ensure that intermediate model updates remain encrypted [67]. Moreover, synthetic data synthesis using generative AI can fill gaps where real data are scarce, sensitive, or inaccessible. When integrated with privacy-preserving pipelines, synthetic data can be shared without compromising patient confidentiality [68]. Additionally, we envision a legally mandated framework under which medical datasets would be preserved securely during patients’ lifetimes and be declassified a few decades later for research use. Such a mechanism, analogous to historical archives in other domains, would balance privacy and scientific value. This will allow future researchers to inherit a comprehensive, ethically sourced repository for studying disease evolution, health trends, and long-range biomedical questions. This means that data governance must not only protect individuals now but also enable transformative discoveries for generations to come. Beyond sheer data volume, incorporating medical knowledge is emerging as a critical enabler for medical foundation models [69]. Knowledge graphs capturing biomedical ontologies, disease–symptom–treatment relationships, and imaging– genomics linkages can guide representation learning toward clinically meaningful concepts. Likewise, retrieval-augmented generation (RAG) allows models to query curated clinical databases, imaging atlases, and knowledge bases during Foundation Model Downstream Application A Downstream Application B Computing Power –GPUs and ecosystems that enable scalable, energy-efficient model training and deployment, with pathways toward next-generation computing platforms. Data/Knowledge –Diverse datasets and knowledge bases that integrate multimodal and multitask information, enabling secure data sharing through federated learning within the healthcare metaverse. Models/Optimization – Innovative architectures trained with self-learning and reinforcement learning, integrating multimodal data into a world model to support causal reasoning, continual learning, and interpretable multitasking. Regulatory Science – Robust evaluation frameworks and regulatory pathways that ensure safe translation and rigorous validation of foundation models for medical imaging. 13 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 inference, providing verifiable yet up-to-date context for predictions [70]. B. Models/Optimization Currently, autoregressive, and diffusion-type models represent two major paradigms. Large language models (LLMs) work via autoregression and are very successful. On the other hand, diffusion-type models have demonstrated exceptional performance in image, video, and multimodal generation. These paradigms can be in contrast: tokenization versus transformation, next-token prediction versus field-based generation, symbolic reasoning versus perceptual changing, and semantic analysis versus manifold learning. Autoregressive models may suffer from exposure bias/error accumulation, but diffusion-type models face computational costs due to iterative sampling, though new solvers and latent-space parameterizations mitigate this issue. Now, a growing trend is to unify both paradigms [71], [72]. Up to today the Transformer architecture remains the mainstream, but breakthroughs are needed for performance boost. Emerging architectures such as Mamba models show the ability to capture long-range context with sub-quadratic complexity [73]. Further improvements like MambaExtend enhance its long-context capabilities via a training-free scaling calibration, enabling up to 32 times longer context windows with minimal computational overhead [74]. Also, differentiable reasoning engines and hybrid neuro-symbolic architecture are emerging to integrate symbolic knowledge with deep learning [75]. Efforts are further made toward brain-inspired architectures that work with novel links, loops, and emerging behaviors [76]. Inspiration from the human brain suggests future artificial neural network architectures with modularity, recurrent connectivity, memory, predictive coding, and cross- modal integration, perception, and reasoning in fast and slow modes. While data-driven techniques offer immense value, they cannot fully replace the role of physics-based models. When the underlying physics is well understood and models accurately approximate reality in a generalizable way, physics-based approaches will consistently outperform purely data-driven ones. Therefore, hybrid architectures that combine physics modeling with deep learning—such as physics-informed foundation models (FMs)—are likely to deliver the most robust and powerful solutions. When grounded in first principles, these FMs become even more foundational. Since a neural architecture is nothing but a computational prototype, we must train it to optimize its parameters and performance, including pre-training, training, and post-training (during test-time/inference) [77]. This is critical but highly nontrivial, as it demands non-convex optimization. Techniques such as reinforcement learning with human or tool feedback and fine-turning in various forms can guide models toward desirable outcomes and meaningful decision-making [78]. Dynamic optimization strategies, including test-time training and adaptation, will be essential to maintain performance under distribution shifts [79]. As medical imaging foundation models evolve, a natural tension lies between generalist models and specialist models. Generalist models benefit from large-scale, multi-modal pretraining, offering transferable representations and a unified inference pipeline across diverse tasks [80]. On the other hand, specialist models, often built through parameter-efficient fine- tuning of a generalist backbone, can achieve higher accuracy, regulatory clarity, and workflow integration for targeted use cases such as lung nodule tracking or cardiac function assessment [81], [82]. Future ecosystems will likely adopt a combined paradigm, where a robust generalist foundation provides shared representations, while specialist derivatives deliver precision, interpretability, and regulatory compliance for mission-critical clinical endpoints. We believe that interactions between specialist and generalist models exemplify bottom-up and top-down methods, defining the dynamics of medical AI. C. Computing Power The rapid advancement of medical imaging foundation models depends critically on high-performance computing resources, with GPU being the main work horse. The pace of computational innovation, from NVIDIA’s cutting-edge architectures to emerging specialized AI accelerators, has enabled ever-larger models, shorter training cycles, and faster inferences, but it also demands investment to stay competitive. Initiatives such as New York’s EmpireAI exemplify forward- looking efforts to enable and democratize access to powerful computing resources, fostering partnerships across the state. As the first state-initiated AI-oriented computational infrastructure, Empire AI [83] brings together nine universities in New York to operate a shared high-performance computing resources. Designed to become a unique academic computing platform, Empire AI has demonstrated strong results with its first-generation Alpha version. The upcoming Beta version represents a major boost, with 7X speedup and 20X acceleration in inference, capable of training multi-trillion- parameter models, Beyond the Beta version, the Buffalo supercomputing facility is expected in 2027 to deliver orders of magnitude greater computational power. Meanwhile, tech giants have active projects like Stargate, highlighting the global best-in-class resources. For medical imaging, forging synergies between universities, hospitals, industry stakeholders, and federate initiatives like EmpireAI will be key to ensuring that cutting-edge computational capacity translates into real-world clinical innovations. In addition to GPUs, emerging computing paradigms hold promises for development of medical imaging foundation models. Quantum computing has the potential to revolutionize foundation models by accelerating large-scale training, enabling quantum-inspired architectures, and unlocking generative and/or discriminative capabilities for complex, high- dimensional problems. As quantum hardware matures, hybrid quantum-classical FMs could become essential, paving the way for breakthroughs in medical imaging and beyond. Neuromorphic computing, inspired by spiking neural networks and event-driven architectures, offers ultra-low-power inference and real-time edge intelligence. Optical computing harnesses photonic interconnects and analog light-based operations to achieve massively parallel, energy-efficient matrix computations beyond the scaling limits of electronic chips. At another frontier, synthetic biological intelligence does IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 14 Fig. 4.2. High-level roadmap for development of medical imaging foundation models underpinned by the four pillars, highlighting how technical, regulatory, and clinical advances converge to enable trustworthy, high-performance AI systems, while ensuring their safe, ethical, and viable deployment for healthcare benefits. computation using engineered cellular or molecular systems, opening a radically new substrate for learning and decision- making under biochemical constraints. As these technologies mature, hybrid computing ecosystems that integrate electronic, photonic, neuromorphic, and biological processors could deliver orders-of-magnitude improvements in speed, energy efficiency, and adaptability. D. Regulatory Science The rapid development of medical imaging foundation models (FMs) has outpaced existing regulatory frameworks, underscoring the urgent need to develop new rules tailored to address their unique challenges. While the U.S. FDA has begun adapting its existing AI/ML-based software regulations, such as predetermined change control plans and Good Machine Learning Practice (GMLP), these general frameworks do not fully account for the unique features of medical foundation models. Unlike traditional task-specific AI systems, foundation models are pre-trained on vast, heterogeneous datasets and can be fine-tuned or prompted for a range of downstream tasks. This introduces new regulatory challenges. Given the mission- critical nature of medical applications, it is essential to develop a dedicated regulatory science strategy for foundation models to address generalizability, explainability, monitoring, and so on. To embed explainability in foundation models, a promising approach is to synergistically combine Chain-of-Thought (CoT) reasoning [84] and causal analysis [85]. CoT reasoning offers a narrative, step-by-step explanation of how a model arrives at a decision, helping users interpret its internal logic. However, these reasoning traces can be post hoc rationalizations rather than genuine causal mechanisms. On the other hand, causal analysis aims to uncover the underlying cause-and-effect relationships driving a model’s predictions. While more rigorous, causal models alone may lack the intuitive transparency needed for clinicians to trust and adopt the system. We advocate that integrating these two complementary approaches by aligning the CoT explanation with the model’s learned or inferred causal structure [86]. In principle, this coupling can produce a dual-layered explainability framework that is both human-interpretable and epistemically sound. Such an approach not only enhances the trustworthiness of medical AI systems but also provides regulators with a principled method to evaluate explainability claims. In the context of foundation models where tasks and data may shift significantly over time, this approach offers a scalable and rigorous way to monitor model reasoning across clinical scenarios. A cornerstone of regulatory oversight is ensuring generalizability under distribution shifts. Prospective multi- institutional benchmarks should stress-test models across raw acquisition, reconstruction, enhancement, and diagnostic tasks to quantify variability introduced by hardware differences, imaging physics, and clinical protocols [87] [88]. Techniques such as domain adaptation, test-time training, and federated or cross-site training reduce sensitivity to site- or device-specific artifacts. Subgroup performance disparities can undermine clinical trust, and can be measured with disaggregated metrics, dataset design through balanced cohort selection, and augmentation for underrepresented groups. Equally important is that AI models should produce well-calibrated confidence estimates [89] and abstain from predictions when uncertainty is high, providing an indicator for potential deterioration of model performance over time. Also, retrieval-augmented generation and tool grounding help ensure that diagnostic claims remain linked to verifiable evidence rather than hallucinated content [90]. Present (2025-2027)Near Future (until 2030)Long Term (beyond 2030) Technical Innovation Regulatory Evolution Clinical Integration Healthcare Systems Closed-Loop Feedback Multimodal Data Transformers Federated Learning Generative AI Mamba / Mamba Variants Test-Time Adaptation Neuro-Symbolic Reasoning Brain-Inspired Architectures Neuromorphic / Optical Computing Automated Multi-Modal Pipelines FDA PCCPs EU AI Act Regulatory Sandboxes Global AI Safety Standards & Guidelines Continuous Post-Market Surveillance Retrospective Validation Single-site Pilots Prospective Multi-Center Trials Explainability (CoT + Causal) Real-Time Auditing, Adapting, & Optimization Hybrid & Other Architectures Digital Twins Healthcare Metaverses Evolving Eco-system of AI Models Quantum Computing? 15 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 Regulatory science treats model deployment as the beginning of oversight. Clinical integration requires continuous monitoring for data drift, adversarial inputs, and performance regressions, coupled with shadow mode evaluations before live deployment. The FDA’s Predetermined Change Control Plans (PCCPs), the EU AI Act, the UK’s MHRA SaMD framework, and the NIST AI Risk Management Framework converge on principles for pre-authorized updates, locked evaluation datasets, rollback procedures, and post-market surveillance. In this monitoring process, transparent governance underpins all regulatory stages. Model cards, data sheets, and stratified performance specifications by site, modality, and patient subgroup create an auditable evidence base. Documenting training datasets and steps, algorithmic changes, calibration methods, and real-world performance metrics supports reproducibility and facilitates global regulatory harmonization. E. Concluding Remarks The convergence of data/knowledge, models/optimization, computing power, and regulatory science is redefining medical imaging foundation models. Further success demands high- quality, multimodal data, innovative architectures, sustainable computing ecosystems, and contemporary regulatory frameworks. Future foundation models will likely combine generalist representations across modalities and tasks with specialist derivatives fine-tuned for high-stakes clinical endpoints, supported by advances in federated learning, privacy-preserving synthetic augmentation, and retrieval- augmented reasoning to break down data silos. At the same time, brain-inspired architectures, physics-informed generative models, and emerging computing paradigms promise major gains in efficiency and capability. Yet, clinical translation will ultimately hinge on rigorous and transparent governance and continuous post-deployment oversight. By uniting technical breakthroughs with ethical and regulatory rigor, medical imaging foundation models can evolve into future healthcare systems, being powerful, trustworthy, and impactful. REFERENCES [1] M. Awais et al., “Foundational Models Defining a New Era in Vision: A Survey and Outlook,” 2023, arXiv. doi: 10.48550/ARXIV.2307.13721. [2] A. M. Turing, “Computing Machinery and Intelligence,” in Parsing the Turing Test, R. Epstein, G. Roberts, and G. Beber, Eds., Dordrecht: Springer Netherlands, 2009, p. 23–65. doi: 10.1007/978-1-4020-6710-5_3. [3] M. Campbell, A. J. Hoane Jr, and F. Hsu, “Deep blue,” Artif. Intell., vol. 134, no. 1–2, p. 57–83, 2002. [4] E. Shortliffe, Computer-based medical consultations: MYCIN, vol. 2. Elsevier, 2012. Accessed: Oct. 17, 2025. [Online]. Available: https://books.google.com/books?hl=zh- CN&lr=&id=i9QXugPQw6oC&oi=fnd&pg=P1&dq= Computer- based+medical+consultations:+MYCIN%5BM%5D.+El sevier,+2012.&ots=Ib7h- 09wQG&sig=KVtV3cRHEDRj9xzLLY3kPDZeu5Y [5] T. Winograd, “Procedures as a representation for data in a computer program for understanding natural language,” 1971. Accessed: Oct. 17, 2025. [Online]. Available: https://apps.dtic.mil/sti/html/tr/AD0721399/ [6] D. A. Ferrucci, “Introduction to ‘this is watson,’” IBM J. Res. Dev., vol. 56, no. 3.4, p. 1–1, 2012. [7] S. M. Stigler, “Gauss and the invention of least squares,” Ann. Stat., p. 465–474, 1981. [8] C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, no. 3, p. 273–297, Sept. 1995, doi: 10.1007/BF00994018. [9] L. Breiman, “Random Forests,” Mach. Learn., vol. 45, no. 1, p. 5–32, Oct. 2001, doi: 10.1023/A:1010933404324. [10] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, p. 2278–2324, 2002. [11] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Adv. Neural Inf. Process. Syst., vol. 25, 2012, Accessed: Oct. 17, 2025. [Online]. Available: https://proceedings.neurips.c/paper/2012/hash/c399862 d3b9d6b76c8436e924a68c45b-Abstract.html [12] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, p. 770–778. Accessed: Oct. 17, 2025. [Online]. Available: http://openaccess.thecvf.com/content_cvpr_2016/html/ He_Deep_Residual_Learning_CVPR_2016_paper.html [13] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, p. 1735–1780, 1997. [14] I. Goodfellow et al., “Generative adversarial networks,” Commun. ACM, vol. 63, no. 11, p. 139–144, Oct. 2020, doi: 10.1145/3422622. [15] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” May 18, 2015, arXiv: arXiv:1505.04597. doi: 10.48550/arXiv.1505.04597. [16] J. Jumper et al., “Highly accurate protein structure prediction with AlphaFold,” nature, vol. 596, no. 7873, p. 583–589, 2021. [17] K. Singhal et al., “Large language models encode clinical knowledge,” Nature, vol. 620, no. 7972, p. 172–180, 2023. [18] R. Bommasani, “On the opportunities and risks of foundation models,” ArXiv Prepr. ArXiv210807258, 2021. [19] “Introducing the Center for Research on Foundation Models (CRFM) | Stanford HAI.” Accessed: Oct. 17, 2025. [Online]. Available: https://hai.stanford.edu/news/introducing-center- research-foundation-models-crfm [20] S. Alfasly et al., “When is a Foundation Model a Foundation Model,” Sept. 14, 2023, arXiv: arXiv:2309.11510. doi: 10.48550/arXiv.2309.11510. 16 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 [21] X. Liu et al., “Self-supervised Learning: Generative or Contrastive,” IEEE Trans. Knowl. Data Eng., p. 1–1, 2021, doi: 10.1109/TKDE.2021.3090866. [22] L. Yang et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” Sept. 27, 2025, arXiv: arXiv:2209.00796. doi: 10.48550/arXiv.2209.00796. [23] A. Kazerouni et al., “Diffusion Models for Medical Image Analysis: A Comprehensive Survey,” June 03, 2023, arXiv: arXiv:2211.07804. doi: 10.48550/arXiv.2211.07804. [24] D. Hein, A. Bozorgpour, D. Merhof, and G. Wang, “Physics-Inspired Generative Models in Medical Imaging: A Review,” Aug. 23, 2024, arXiv: arXiv:2407.10856. doi: 10.48550/arXiv.2407.10856. [25] S. Longpre et al., “The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources,” Feb. 17, 2025, arXiv: arXiv:2406.16746. doi: 10.48550/arXiv.2406.16746. [26] W. Zhou et al., “The security of using large language models: A survey with emphasis on ChatGPT,” IEEECAA J. Autom. Sin., 2024, Accessed: Oct. 17, 2025. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10751746/ [27] W. X. Zhao et al., “A survey of large language models,” ArXiv Prepr. ArXiv230318223, vol. 1, no. 2, 2023, Accessed: Oct. 17, 2025. [Online]. Available: https://w.researchgate.net/profile/Tang-Tianyi- 3/publication/369740832_A_Survey_of_Large_Langua ge_Models/links/665fd2e3637e4448a37d281/A- Survey-of-Large-Language-Models.pdf [28] I. A. Scott and G. Zuccon, “The new paradigm in machine learning – foundation models, large language models and beyond: a primer for physicians,” Intern. Med. J., vol. 54, no. 5, p. 705–715, May 2024, doi: 10.1111/imj.16393. [29] J. Yang et al., “Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond,” ACM Trans. Knowl. Discov. Data, vol. 18, no. 6, p. 1–32, July 2024, doi: 10.1145/3649506. [30] D. Truhn, J.-N. Eckardt, D. Ferber, and J. N. Kather, “Large language models and multimodal foundation models for precision oncology,” NPJ Precis. Oncol., vol. 8, no. 1, p. 72, 2024. [31] B. Azad et al., “Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision,” Oct. 28, 2023, arXiv: arXiv:2310.18689. doi: 10.48550/arXiv.2310.18689. [32] S.-C. Huang, M. Jensen, S. Yeung-Levy, M. P. Lungren, H. Poon, and A. S. Chaudhari, “Multimodal Foundation Models for Medical Imaging-A Systematic Review and Implementation Guidelines,” medRxiv, p. 2024–10, 2024. [33] J. S. Ryu, H. Kang, Y. Chu, and S. Yang, “Vision- language foundation models for medical imaging: a review of current practices and innovations,” Biomed. Eng. Lett., vol. 15, no. 5, p. 809–830, Sept. 2025, doi: 10.1007/s13534-025-00484-6. [34] K. Sun et al., “Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions,” Dec. 03, 2024, arXiv: arXiv:2412.02621. doi: 10.48550/arXiv.2412.02621. [35] S. Zhang and D. Metaxas, “On the Challenges and Perspectives of Foundation Models for Medical Image Analysis,” Nov. 21, 2023, arXiv: arXiv:2306.05705. doi: 10.48550/arXiv.2306.05705. [36] J. P. Huix, A. R. Ganeshan, J. F. Haslum, M. Söderberg, C. Matsoukas, and K. Smith, “Are Natural Domain Foundation Models Useful for Medical Image Classification?,” in 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA: IEEE, Jan. 2024, p. 7619–7628. doi: 10.1109/WACV57701.2024.00746. [37] V. van Veldhuizen et al., “Foundation Models in Medical Imaging -- A Review and Outlook,” June 16, 2025, arXiv: arXiv:2506.09095. doi: 10.48550/arXiv.2506.09095. [38] M. O. Khan, M. M. Afzal, S. Mirza, and Y. Fang, “How Fair are Medical Imaging Foundation Models?,” in Proceedings of the 3rd Machine Learning for Health Symposium, PMLR, Dec. 2023, p. 217–231. Accessed: Oct. 17, 2025. [Online]. Available: https://proceedings.mlr.press/v225/khan23a.html [39] A. Vaswani et al., “Attention is All you Need”. [40] C. Niu and G. Wang, “Unsupervised contrastive learning based transformer for lung nodule detection,” Phys. Med. Biol., vol. 67, no. 20, p. 204001, Oct. 2022, doi: 10.1088/1361-6560/ac92ba. [41] A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” June 03, 2021, arXiv: arXiv:2010.11929. doi: 10.48550/arXiv.2010.11929. [42] Z. Liu et al., “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” Aug. 17, 2021, arXiv: arXiv:2103.14030. doi: 10.48550/arXiv.2103.14030. [43] Y. Li, H. Mao, R. Girshick, and K. He, “Exploring Plain Vision Transformer Backbones for Object Detection,” in Computer Vision – ECCV 2022, vol. 13669, S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds., in Lecture Notes in Computer Science, vol. 13669. , Cham: Springer Nature Switzerland, 2022, p. 280–296. doi: 10.1007/978-3-031-20077-9_17. [44] A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving Language Understanding by Generative Pre-Training”. [45] S. Bai et al., “Qwen2.5-VL Technical Report,” Feb. 19, 2025, arXiv: arXiv:2502.13923. doi: 10.48550/arXiv.2502.13923. [46] L. Blankemeier et al., “Merlin: A Vision Language Foundation Model for 3D Computed Tomography,” June 10, 2024, arXiv: arXiv:2406.06512. doi: 10.48550/arXiv.2406.06512. [47] N. Shazeer et al., “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” Jan. 23, 17 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 2017, arXiv: arXiv:1701.06538. doi: 10.48550/arXiv.1701.06538. [48] D. Dai et al., “DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models,” Jan. 11, 2024, arXiv: arXiv:2401.06066. doi: 10.48550/arXiv.2401.06066. [49] R. Child, S. Gray, A. Radford, and I. Sutskever, “Generating Long Sequences with Sparse Transformers,” Apr. 23, 2019, arXiv: arXiv:1904.10509. doi: 10.48550/arXiv.1904.10509. [50] K. Choromanski et al., “Rethinking Attention with Performers,” Nov. 19, 2022, arXiv: arXiv:2009.14794. doi: 10.48550/arXiv.2009.14794. [51] A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, “Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention,” Aug. 31, 2020, arXiv: arXiv:2006.16236. doi: 10.48550/arXiv.2006.16236. [52] S. Wang, B. Z. Li, M. Khabsa, H. Fang, and H. Ma, “Linformer: Self-Attention with Linear Complexity,” June 14, 2020, arXiv: arXiv:2006.04768. doi: 10.48550/arXiv.2006.04768. [53] J. Ainslie, J. Lee-Thorp, M. de Jong, Y. Zemlyanskiy, F. Lebrón, and S. Sanghai, “GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints,” Dec. 23, 2023, arXiv: arXiv:2305.13245. doi: 10.48550/arXiv.2305.13245. [54] N. Shazeer, “Fast Transformer Decoding: One Write- Head is All You Need,” Nov. 06, 2019, arXiv: arXiv:1911.02150. doi: 10.48550/arXiv.1911.02150. [55] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, p. 84–90, May 2017, doi: 10.1145/3065386. [56] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” Dec. 10, 2015, arXiv: arXiv:1512.03385. doi: 10.48550/arXiv.1512.03385. [57] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “CBAM: Convolutional Block Attention Module,” in Computer Vision – ECCV 2018, vol. 11211, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds., in Lecture Notes in Computer Science, vol. 11211. , Cham: Springer International Publishing, 2018, p. 3–19. doi: 10.1007/978-3-030-01234-2_1. [58] I. Bello, B. Zoph, Q. Le, A. Vaswani, and J. Shlens, “Attention Augmented Convolutional Networks,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South): IEEE, Oct. 2019, p. 3285–3294. doi: 10.1109/ICCV.2019.00338. [59] H. Wu et al., “CvT: Introducing Convolutions to Vision Transformers,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada: IEEE, Oct. 2021, p. 22–31. doi: 10.1109/ICCV48922.2021.00009. [60] A. Sherstinsky, “Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network,” Phys. Nonlinear Phenom., vol. 404, p. 132306, Mar. 2020, doi: 10.1016/j.physd.2019.132306. [61] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling,” Dec. 11, 2014, arXiv: arXiv:1412.3555. doi: 10.48550/arXiv.1412.3555. [62] A. Gu, K. Goel, and C. Ré, “Efficiently Modeling Long Sequences with Structured State Spaces,” Aug. 05, 2022, arXiv: arXiv:2111.00396. doi: 10.48550/arXiv.2111.00396. [63] A. Gu and T. Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” May 31, 2024, arXiv: arXiv:2312.00752. doi: 10.48550/arXiv.2312.00752. [64] B. Peng et al., “RWKV: Reinventing RNNs for the Transformer Era,” Dec. 11, 2023, arXiv: arXiv:2305.13048. doi: 10.48550/arXiv.2305.13048. [65] D. Y. Fu, T. Dao, K. K. Saab, A. W. Thomas, A. Rudra, and C. Ré, “Hungry Hungry Hippos: Towards Language Modeling with State Space Models,” Apr. 29, 2023, arXiv: arXiv:2212.14052. doi: 10.48550/arXiv.2212.14052. [66] Y. Sun et al., “Retentive Network: A Successor to Transformer for Large Language Models,” Aug. 09, 2023, arXiv: arXiv:2307.08621. doi: 10.48550/arXiv.2307.08621. [67] Y. Yue and Z. Li, “MedMamba: Vision Mamba for Medical Image Classification,” Sept. 29, 2024, arXiv: arXiv:2403.03849. doi: 10.48550/arXiv.2403.03849. [68] Z. Chen et al., “LangMamba: A Language-driven Mamba Framework for Low-dose CT Denoising with Vision-language Models,” July 08, 2025, arXiv: arXiv:2507.06140. doi: 10.48550/arXiv.2507.06140. [69] D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” Dec. 10, 2022, arXiv: arXiv:1312.6114. doi: 10.48550/arXiv.1312.6114. [70] I. Higgins et al., “β-VAE: LEARNING BASIC VISUAL CONCEPTS WITH A CONSTRAINED VARIATIONAL FRAMEWORK,” 2017. [71] A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural Discrete Representation Learning,” May 30, 2018, arXiv: arXiv:1711.00937. doi: 10.48550/arXiv.1711.00937. [72] Z. Zheng et al., “Open-Sora: Democratizing Efficient Video Production for All,” Dec. 29, 2024, arXiv: arXiv:2412.20404. doi: 10.48550/arXiv.2412.20404. [73] I. J. Goodfellow et al., “Generative Adversarial Networks,” June 10, 2014, arXiv: arXiv:1406.2661. doi: 10.48550/arXiv.1406.2661. [74] A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” Jan. 07, 2016, arXiv: arXiv:1511.06434. doi: 10.48550/arXiv.1511.06434. [75] T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive Growing of GANs for Improved Quality, Stability, and Variation,” Feb. 26, 2018, arXiv: arXiv:1710.10196. doi: 10.48550/arXiv.1710.10196. [76] T. Karras, S. Laine, and T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” Mar. 29, 2019, arXiv: arXiv:1812.04948. doi: 10.48550/arXiv.1812.04948. 18 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 [77] A. Brock, J. Donahue, and K. Simonyan, “Large Scale GAN Training for High Fidelity Natural Image Synthesis,” Feb. 25, 2019, arXiv: arXiv:1809.11096. doi: 10.48550/arXiv.1809.11096. [78] S. Dayarathna, K. T. Islam, S. Uribe, G. Yang, M. Hayat, and Z. Chen, “Deep learning based synthesis of MRI, CT and PET: Review and analysis,” Med. Image Anal., vol. 92, p. 103046, 2024, doi: https://doi.org/10.1016/j.media.2023.103046. [79] C. You et al., “CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE),” IEEE Trans. Med. Imaging, vol. 39, no. 1, p. 188–203, Jan. 2020, doi: 10.1109/TMI.2019.2922960. [80] Q. Yang et al., “Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss,” IEEE Trans. Med. Imaging, vol. 37, no. 6, p. 1348–1357, June 2018, doi: 10.1109/TMI.2018.2827462. [81] C. Niu et al., “Low-Dimensional Manifold-Constrained Disentanglement Network for Metal Artifact Reduction,” IEEE Trans. Radiat. Plasma Med. Sci., vol. 6, no. 6, p. 656–666, July 2022, doi: 10.1109/TRPMS.2021.3122071. [82] C. Niu et al., “Noise Entangled GAN For Low-Dose CT Simulation,” Feb. 18, 2021, arXiv: arXiv:2102.09615. doi: 10.48550/arXiv.2102.09615. [83] J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models,” Dec. 16, 2020, arXiv: arXiv:2006.11239. doi: 10.48550/arXiv.2006.11239. [84] L. Yang et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” Dec. 02, 2024, arXiv: arXiv:2209.00796. doi: 10.48550/arXiv.2209.00796. [85] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency Models,” May 31, 2023, arXiv: arXiv:2303.01469. doi: 10.48550/arXiv.2303.01469. [86] D. Hein, A. Bozorgpour, D. Merhof, and G. Wang, “Physics-Inspired Generative Models in Medical Imaging: A Review,” Aug. 23, 2024, arXiv: arXiv:2407.10856. doi: 10.48550/arXiv.2407.10856. [87] T. B. Brown et al., “Language Models are Few-Shot Learners,” July 22, 2020, arXiv: arXiv:2005.14165. doi: 10.48550/arXiv.2005.14165. [88] A. van den Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel Recurrent Neural Networks,” Aug. 19, 2016, arXiv: arXiv:1601.06759. doi: 10.48550/arXiv.1601.06759. [89] A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt, A. Graves, and K. Kavukcuoglu, “Conditional Image Generation with PixelCNN Decoders,” June 18, 2016, arXiv: arXiv:1606.05328. doi: 10.48550/arXiv.1606.05328. [90] A. Ramesh et al., “Zero-Shot Text-to-Image Generation,” Feb. 26, 2021, arXiv: arXiv:2102.12092. doi: 10.48550/arXiv.2102.12092. [91] P. Sun et al., “Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation,” June 10, 2024, arXiv: arXiv:2406.06525. doi: 10.48550/arXiv.2406.06525. [92] X. Wang et al., “Emu3: Next-Token Prediction is All You Need,” Sept. 27, 2024, arXiv: arXiv:2409.18869. doi: 10.48550/arXiv.2409.18869. [93] A. van den Oord, Y. Li, and O. Vinyals, “Representation Learning with Contrastive Predictive Coding,” Jan. 22, 2019, arXiv: arXiv:1807.03748. doi: 10.48550/arXiv.1807.03748. [94] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Simple Framework for Contrastive Learning of Visual Representations,” July 01, 2020, arXiv: arXiv:2002.05709. doi: 10.48550/arXiv.2002.05709. [95] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum Contrast for Unsupervised Visual Representation Learning,” Mar. 23, 2020, arXiv: arXiv:1911.05722. doi: 10.48550/arXiv.1911.05722. [96] I. Misra and L. van der Maaten, “Self-Supervised Learning of Pretext-Invariant Representations,” Dec. 04, 2019, arXiv: arXiv:1912.01991. doi: 10.48550/arXiv.1912.01991. [97] J.-B. Grill et al., “Bootstrap your own latent: A new approach to self-supervised Learning,” Sept. 10, 2020, arXiv: arXiv:2006.07733. doi: 10.48550/arXiv.2006.07733. [98] X. Chen and K. He, “Exploring Simple Siamese Representation Learning,” Nov. 20, 2020, arXiv: arXiv:2011.10566. doi: 10.48550/arXiv.2011.10566. [99] M. Caron et al., “Emerging Properties in Self-Supervised Vision Transformers,” May 24, 2021, arXiv: arXiv:2104.14294. doi: 10.48550/arXiv.2104.14294. [100] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, “Unsupervised Learning of Visual Features by Contrasting Cluster Assignments,” Jan. 08, 2021, arXiv: arXiv:2006.09882. doi: 10.48550/arXiv.2006.09882. [101] M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep Clustering for Unsupervised Learning of Visual Features,” Mar. 18, 2019, arXiv: arXiv:1807.05520. doi: 10.48550/arXiv.1807.05520. [102] Y. M. Asano, C. Rupprecht, and A. Vedaldi, “Self- labelling via simultaneous clustering and representation learning,” Feb. 19, 2020, arXiv: arXiv:1911.05371. doi: 10.48550/arXiv.1911.05371. [103] A. Bardes, J. Ponce, and Y. LeCun, “VICReg: Variance- Invariance-Covariance Regularization for Self- Supervised Learning,” Jan. 28, 2022, arXiv: arXiv:2105.04906. doi: 10.48550/arXiv.2105.04906. [104] C. Niu, W. Xia, H. Shan, and G. Wang, “Information- Maximized Soft Variable Discretization for Self- Supervised Image Representation Learning,” Jan. 07, 2025, arXiv: arXiv:2501.03469. doi: 10.48550/arXiv.2501.03469. [105] C. Niu et al., “Noise suppression with similarity-based self-supervised deep learning,” IEEE Trans. Med. Imaging, vol. 42, no. 6, p. 1590–1602, 2022. [106] S.-C. Huang, A. Pareek, M. Jensen, M. P. Lungren, S. Yeung, and A. S. Chaudhari, “Self-supervised learning for medical image classification: a systematic review and 19 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 implementation guidelines,” Npj Digit. Med., vol. 6, no. 1, p. 74, Apr. 2023, doi: 10.1038/s41746-023-00811-0. [107] A. Radford et al., “Learning Transferable Visual Models From Natural Language Supervision,” Feb. 26, 2021, arXiv: arXiv:2103.00020. doi: 10.48550/arXiv.2103.00020. [108] C. Jia et al., “Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision,” June 11, 2021, arXiv: arXiv:2102.05918. doi: 10.48550/arXiv.2102.05918. [109] Z. Zhao et al., “CLIP in medical imaging: A survey,” Med. Image Anal., vol. 102, p. 103551, May 2025, doi: 10.1016/j.media.2025.103551. [110] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” May 24, 2019, arXiv: arXiv:1810.04805. doi: 10.48550/arXiv.1810.04805. [111] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked Autoencoders Are Scalable Vision Learners,” Dec. 19, 2021, arXiv: arXiv:2111.06377. doi: 10.48550/arXiv.2111.06377. [112] Y. LeCun, “A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27”. [113] M. Assran et al., “Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture,” Apr. 13, 2023, arXiv: arXiv:2301.08243. doi: 10.48550/arXiv.2301.08243. [114] A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, “data2vec: A General Framework for Self- supervised Learning in Speech, Vision and Language,” Oct. 25, 2022, arXiv: arXiv:2202.03555. doi: 10.48550/arXiv.2202.03555. [115] D. C. Knill and A. Pouget, “The Bayesian brain: the role of uncertainty in neural coding and computation,” TRENDS Neurosci., vol. 27, no. 12, p. 712–719, 2004. [116] K. Friston, “The free-energy principle: a unified brain theory?,” Nat. Rev. Neurosci., vol. 11, no. 2, p. 127– 138, 2010. [117] L. Ouyang et al., “Training language models to follow instructions with human feedback,” Mar. 04, 2022, arXiv: arXiv:2203.02155. doi: 10.48550/arXiv.2203.02155. [118] P. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Feb. 17, 2023, arXiv: arXiv:1706.03741. doi: 10.48550/arXiv.1706.03741. [119] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” Aug. 28, 2017, arXiv: arXiv:1707.06347. doi: 10.48550/arXiv.1707.06347. [120] N. Stiennon et al., “Learning to summarize from human feedback,” Feb. 15, 2022, arXiv: arXiv:2009.01325. doi: 10.48550/arXiv.2009.01325. [121] R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, “Direct Preference Optimization: Your Language Model is Secretly a Reward Model,” July 29, 2024, arXiv: arXiv:2305.18290. doi: 10.48550/arXiv.2305.18290. [122] J. Hong, N. Lee, and J. Thorne, “ORPO: Monolithic Preference Optimization without Reference Model,” Mar. 14, 2024, arXiv: arXiv:2403.07691. doi: 10.48550/arXiv.2403.07691. [123] O. Banerjee, H.-Y. Zhou, S. Adithan, S. Kwak, K. Wu, and P. Rajpurkar, “Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation,” June 14, 2024, arXiv: arXiv:2406.06496. doi: 10.48550/arXiv.2406.06496. [124] S. A. Nahian and R. Kavuluru, “RadQA-DPO: A Radiology Question Answering System with Encoder- Decoder Models Enhanced by Direct Preference Optimization”. [125] Z. Shao et al., “DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models,” Apr. 27, 2024, arXiv: arXiv:2402.03300. doi: 10.48550/arXiv.2402.03300. [126] Q. Yu et al., “DAPO: An Open-Source LLM Reinforcement Learning System at Scale,” May 20, 2025, arXiv: arXiv:2503.14476. doi: 10.48550/arXiv.2503.14476. [127] C. Zheng et al., “Group Sequence Policy Optimization,” July 28, 2025, arXiv: arXiv:2507.18071. doi: 10.48550/arXiv.2507.18071. [128] H. Lightman et al., “Let’s Verify Step by Step,” May 31, 2023, arXiv: arXiv:2305.20050. doi: 10.48550/arXiv.2305.20050. [129] S. Yao et al., “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” Dec. 03, 2023, arXiv: arXiv:2305.10601. doi: 10.48550/arXiv.2305.10601. [130] Z. Chen, Y. Deng, H. Yuan, K. Ji, and Q. Gu, “Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models,” June 14, 2024, arXiv: arXiv:2401.01335. doi: 10.48550/arXiv.2401.01335. [131] H. Wang et al., “Offline Reinforcement Learning for LLM Multi-Step Reasoning,” Dec. 25, 2024, arXiv: arXiv:2412.16145. doi: 10.48550/arXiv.2412.16145. [132] Y. Lai et al., “Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models,” 2025, arXiv. doi: 10.48550/ARXIV.2503.13939. [133] C. Niu and G. Wang, “Reasoning Language Model for Personalized Lung Cancer Screening,” Sept. 07, 2025, arXiv: arXiv:2509.06169. doi: 10.48550/arXiv.2509.06169. [134] J. Lee et al., “BioBERT: a pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, p. 1234–1240, Feb. 2020, doi: 10.1093/bioinformatics/btz682. [135] J. Ma, Y. He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nat. Commun., vol. 15, no. 1, p. 654, Jan. 2024, doi: 10.1038/s41467- 024-44824-z. [136] C. Niu et al., “Medical multimodal multitask foundation model for lung cancer screening,” Nat. Commun., vol. 16, no. 1, p. 1523, Feb. 2025, doi: 10.1038/s41467-025- 56822-w. 20 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 [137] J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye, “A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications,” Jan. 20, 2020, arXiv: arXiv:2001.06937. doi: 10.48550/arXiv.2001.06937. [138] W. Xia, Y. Shi, C. Niu, W. Cong, and G. Wang, “Diffusion Prior Regularized Iterative Reconstruction for Low-dose CT,” Oct. 10, 2023, arXiv: arXiv:2310.06949. doi: 10.48550/arXiv.2310.06949. [139] C. Niu, C. Wiedeman, M. Li, J. S. Maltz, and G. Wang, “3D Photon Counting CT Image Super-Resolution Using Conditional Diffusion Model,” Aug. 22, 2024, arXiv: arXiv:2408.15283. doi: 10.48550/arXiv.2408.15283. [140] X. Liu, Y. Xie, J. Cheng, S. Diao, S. Tan, and X. Liang, “Diffusion Probabilistic Priors for Zero-Shot Low-Dose CT Image Denoising,” July 14, 2023, arXiv: arXiv:2305.15887. doi: 10.48550/arXiv.2305.15887. [141] W. Xia et al., “Dual-Domain Denoising Diffusion Probabilistic Model for Metal Artifact Reduction,” IEEE Trans. Radiat. Plasma Med. Sci., p. 1–1, 2025, doi: 10.1109/TRPMS.2025.3582528. [142] E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” Oct. 16, 2021, arXiv: arXiv:2106.09685. doi: 10.48550/arXiv.2106.09685. [143] “DeepSeek_R1.” [144] Y. Zhao et al., “PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel,” Sept. 12, 2023, arXiv: arXiv:2304.11277. doi: 10.48550/arXiv.2304.11277. [145] R. Y. Aminabadi et al., “DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale,” June 30, 2022, arXiv: arXiv:2207.00032. doi: 10.48550/arXiv.2207.00032. [146] E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” Oct. 16, 2021, arXiv: arXiv:2106.09685. doi: 10.48550/arXiv.2106.09685. [147] Y. Mao et al., “A Survey on LoRA of Large Language Models,” Front. Comput. Sci., vol. 19, no. 7, p. 197605, July 2025, doi: 10.1007/s11704-024-40663-9. [148] T. Dao, “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,” July 17, 2023, arXiv: arXiv:2307.08691. doi: 10.48550/arXiv.2307.08691. [149] P.-L. Hsu et al., “Liger Kernel: Efficient Triton Kernels for LLM Training,” Jan. 24, 2025, arXiv: arXiv:2410.10989. doi: 10.48550/arXiv.2410.10989. [150] P. Micikevicius et al., “Mixed Precision Training,” Feb. 15, 2018, arXiv: arXiv:1710.03740. doi: 10.48550/arXiv.1710.03740. [151] V. Korthikanti et al., “REDUCING ACTIVATION RECOMPUTATION IN LARGE TRANSFORMER MODELS”. [152] J. Lamy-Poirier, “Layered gradient accumulation and modular pipeline parallelism: fast and efficient training of large language models,” June 04, 2021, arXiv: arXiv:2106.02679. doi: 10.48550/arXiv.2106.02679. [153] G. Hinton, O. Vinyals, and J. Dean, “Distilling the Knowledge in a Neural Network,” Mar. 09, 2015, arXiv: arXiv:1503.02531. doi: 10.48550/arXiv.1503.02531. [154] S. Li et al., “Evaluating Quantized Large Language Models,” June 06, 2024, arXiv: arXiv:2402.18158. doi: 10.48550/arXiv.2402.18158. [155] W. Kwon et al., “Efficient Memory Management for Large Language Model Serving with PagedAttention,” Sept. 12, 2023, arXiv: arXiv:2309.06180. doi: 10.48550/arXiv.2309.06180. [156] L. Zheng et al., “SGLang: Efficient Execution of Structured Language Model Programs,” June 06, 2024, arXiv: arXiv:2312.07104. doi: 10.48550/arXiv.2312.07104. [157] Z. Ramzi, G. R. Chaithya, J.-L. Starck, and P. Ciuciu, “NC-PDNet: A density-compensated unrolled network for 2D and 3D non-Cartesian MRI reconstruction,” IEEE Trans. Med. Imaging, vol. 41, no. 7, p. 1625–1638, 2022. [158] K. Hammernik et al., “Learning a variational network for reconstruction of accelerated MRI data,” Magn. Reson. Med., vol. 79, no. 6, p. 3055–3071, June 2018, doi: 10.1002/mrm.26977. [159] J. Adler and O. Öktem, “Learned primal-dual reconstruction,” IEEE Trans. Med. Imaging, vol. 37, no. 6, p. 1322–1332, 2018. [160] “Pruning Unrolled Networks (PUN) at Initialization for MRI Reconstruction Improves Generalization.” Accessed: Oct. 17, 2025. [Online]. Available: https://arxiv.org/html/2412.18668v1 [161] K. Zhang et al., “Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis,” Mach. Intell. Res., vol. 20, no. 6, p. 822–836, Dec. 2023, doi: 10.1007/s11633-023-1466-0. [162] L. Zuo et al., “Unsupervised MR harmonization by learning disentangled representations using information bottleneck theory,” NeuroImage, vol. 243, p. 118569, 2021. [163] Y. Liu et al., “Imaging foundation model for universal enhancement of non-ideal measurement CT,” Feb. 25, 2025, arXiv: arXiv:2410.01591. doi: 10.48550/arXiv.2410.01591. [164] S. Arridge, P. Maass, O. Öktem, and C.-B. Schönlieb, “Solving inverse problems using data-driven models,” Acta Numer., vol. 28, p. 1–174, 2019. [165] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution,” Mar. 27, 2016, arXiv: arXiv:1603.08155. doi: 10.48550/arXiv.1603.08155. [166] M. Terris, S. Hurault, M. Song, and J. Tachella, “Reconstruct Anything Model: a lightweight foundation model for computational imaging,” Sept. 27, 2025, arXiv: arXiv:2503.08915. doi: 10.48550/arXiv.2503.08915. [167] H. Wang et al., “Towards Unified CT Reconstruction: Federated Metadata Learning With Personalized Condition-Modulated iRadonMAP,” IEEE Trans. Radiat. Plasma Med. Sci., 2025, Accessed: Oct. 17, 2025. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/11017337/ [168] J. He, Y. Wang, and J. Ma, “Radon inversion via deep learning,” IEEE Trans. Med. Imaging, vol. 39, no. 6, p. 2076–2087, 2020. [169] L. Fu and B. De Man, “A hierarchical approach to deep learning and its application to tomographic 21 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 reconstruction,” in 15th international meeting on fully three-dimensional image reconstruction in radiology and nuclear medicine, SPIE, 2019, p. 1107202. Accessed: Nov. 22, 2025. [Online]. Available: https://w.spiedigitallibrary.org/conference- proceedings-of-spie/11072/1107202/A-hierarchical- approach-to-deep-learning-and-its-application- to/10.1117/12.2534615.short [170] B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen, and M. S. Rosen, “Image reconstruction by domain-transform manifold learning,” Nature, vol. 555, no. 7697, p. 487– 492, 2018. [171] P. Jain, P. Bv, and N. Sinha, “Novel Complex AUTOMAP for Accelerated MRI,” in Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing, Gandhinagar India: ACM, Dec. 2022, p. 1–9. doi: 10.1145/3571600.3571636. [172] H. Chung, S. Lee, and J. C. Ye, “Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems,” Feb. 19, 2024, arXiv: arXiv:2303.05754. doi: 10.48550/arXiv.2303.05754. [173] J. He, B. Li, G. Yang, and Z. Liu, “Blaze3DM: Marry Triplane Representation with Diffusion for 3D Medical Inverse Problem Solving,” May 24, 2024, arXiv: arXiv:2405.15241. doi: 10.48550/arXiv.2405.15241. [174] W. Xia, C. Niu, and G. Wang, “Tomographic Foundation Model -- FORCE: Flow-Oriented Reconstruction Conditioning Engine,” June 02, 2025, arXiv: arXiv:2506.02149. doi: 10.48550/arXiv.2506.02149. [175] S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg, “Plug-and-play priors for model based reconstruction,” in 2013 IEEE global conference on signal and information processing, IEEE, 2013, p. 945–948. Accessed: Oct. 17, 2025. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/6737048/ [176] J.-C. Pesquet, A. Repetti, M. Terris, and Y. Wiaux, “Learning Maximally Monotone Operators for Image Recovery,” SIAM J. Imaging Sci., vol. 14, no. 3, p. 1206–1237, Jan. 2021, doi: 10.1137/20M1387961. [177] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Phys. Nonlinear Phenom., vol. 60, no. 1–4, p. 259–268, 1992. [178] M. Stephane, “A wavelet tour of signal processing.” Elsevier, 1999. Accessed: Oct. 17, 2025. [Online]. Available: https://w.sciencedirect.com/book/9780123743701/a- wavelet-tour-of-signal-processing [179] E. Tiu, E. Talius, P. Patel, C. P. Langlotz, A. Y. Ng, and P. Rajpurkar, “Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning,” Nat. Biomed. Eng., vol. 6, no. 12, p. 1399– 1406, 2022. [180] S. Pai et al., “Foundation model for cancer imaging biomarkers,” Nat. Mach. Intell., vol. 6, no. 3, p. 354– 367, 2024. [181] H. Xu et al., “A whole-slide foundation model for digital pathology from real-world data,” Nature, vol. 630, no. 8015, p. 181–188, 2024. [182] H. Yu, Q. Wang, X. Zhou, L. Gu, and Z. Zhao, “Deep weighted survival neural networks to survival risk prediction,” Complex Intell. Syst., vol. 11, no. 1, p. 41, Jan. 2025, doi: 10.1007/s40747-024-01670-2. [183] T. Vu et al., “Tabular Foundation Model for Breast Cancer Prognosis using Gene Expression Data,” Oct. 05, 2025, medRxiv. doi: 10.1101/2025.10.03.25337265. [184] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, p. 3431–3440. Accessed: Nov. 22, 2025. [Online]. Available: http://openaccess.thecvf.com/content_cvpr_2015/html/L ong_Fully_Convolutional_Networks_2015_CVPR_pap er.html [185] J. Ma, Y. He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nat. Commun., vol. 15, no. 1, p. 654, Jan. 2024, doi: 10.1038/s41467- 024-44824-z. [186] S. Noh and B.-D. Lee, “A narrative review of foundation models for medical image segmentation: zero-shot performance evaluation on diverse modalities,” Quant. Imaging Med. Surg., vol. 15, no. 6, p. 5825–5858, June 2025, doi: 10.21037/qims-2024-2826. [187] A. Kirillov et al., “Segment Anything,” Apr. 05, 2023, arXiv: arXiv:2304.02643. doi: 10.48550/arXiv.2304.02643. [188] K. Fan, L. Liang, H. Li, W. Situ, W. Zhao, and G. Li, “Research on Medical Image Segmentation Based on SAM and Its Future Prospects,” Bioengineering, vol. 12, no. 6, p. 608, 2025. [189] J. Ma, Y. He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nat. Commun., vol. 15, no. 1, p. 654, 2024. [190] W. Lei, W. Xu, K. Li, X. Zhang, and S. Zhang, “MedLSAM: Localize and segment anything model for 3D CT images,” Med. Image Anal., vol. 99, p. 103370, 2025. [191] S. Gong et al., “3dsam-adapter: Holistic adaptation of sam from 2d to 3d for promptable medical image segmentation,” ArXiv E-Prints, p. arXiv-2306, 2023. [192] J. Sun et al., “Medical image analysis using improved SAM-Med2D: segmentation and classification perspectives,” BMC Med. Imaging, vol. 24, no. 1, p. 241, Sept. 2024, doi: 10.1186/s12880-024-01401-6. [193] Z. Yan et al., “SAMed-2: Selective Memory Enhanced Medical Segment Anything Model,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2025, vol. 15972, J. C. Gee, D. C. Alexander, J. Hong, J. E. Iglesias, C. H. Sudre, A. Venkataraman, P. Golland, J. H. Kim, and J. Park, Eds., in Lecture Notes in Computer Science, vol. 15972. , Cham: Springer Nature Switzerland, 2026, p. 540–550. doi: 10.1007/978-3- 032-05169-1_52. [194] G. Deng et al., “SAM-U: Multi-box Prompts Triggered Uncertainty Estimation for Reliable SAM in Medical Image,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 Workshops, vol. 14394, J. Woo, A. Hering, W. Silva, X. Li, H. Fu, X. Liu, 22 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 F. Xing, S. Purushotham, T. S. Mathai, P. Mukherjee, M. De Grauw, R. Beets Tan, V. Corbetta, E. Kotter, M. Reyes, C. F. Baumgartner, Q. Li, R. Leahy, B. Dong, H. Chen, Y. Huo, J. Lv, X. Xu, X. Li, D. Mahapatra, L. Cheng, C. Petitjean, and B. Presles, Eds., in Lecture Notes in Computer Science, vol. 14394. , Cham: Springer Nature Switzerland, 2023, p. 368–377. doi: 10.1007/978-3-031-47425-5_33. [195] Y. Tian, M. Shi, X. Zhang, B. Zhang, M. Wang, and Y. Shi, “Assisting embodied AI: a survey of 3D segmentation models for medical CT images: Y. Tian et al.,” CCF Trans. Pervasive Comput. Interact., p. 1–22, 2025. [196] J. Cox et al., “BrainSegFounder: Towards 3D foundation models for neuroimage segmentation,” Med. Image Anal., vol. 97, p. 103301, 2024. [197] B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee, “Symmetric diffeomorphic image registration with cross- correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Med. Image Anal., vol. 12, no. 1, p. 26–41, 2008. [198] G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, and A. V. Dalca, “VoxelMorph: A Learning Framework for Deformable Medical Image Registration,” IEEE Trans. Med. Imaging, vol. 38, no. 8, p. 1788–1800, Aug. 2019, doi: 10.1109/TMI.2019.2897538. [199] J. Chen, E. C. Frey, Y. He, W. P. Segars, Y. Li, and Y. Du, “TransMorph: Transformer for unsupervised medical image registration,” Med. Image Anal., vol. 82, p. 102615, Nov. 2022, doi: 10.1016/j.media.2022.102615. [200] M. Hoffmann, B. Billot, D. N. Greve, J. E. Iglesias, B. Fischl, and A. V. Dalca, “SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images,” IEEE Trans. Med. Imaging, vol. 41, no. 3, p. 543–558, Mar. 2022, doi: 10.1109/TMI.2021.3116879. [201] M. P. Heinrich and L. Hansen, “Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation,” Feb. 28, 2022, arXiv: arXiv:2203.00046. doi: 10.48550/arXiv.2203.00046. [202] L. Tian et al., “uniGradICON: A Foundation Model for Medical Image Registration,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, vol. 15002, M. G. Linguraru, Q. Dou, A. Feragen, S. Giannarou, B. Glocker, K. Lekadir, and J. A. Schnabel, Eds., in Lecture Notes in Computer Science, vol. 15002. , Cham: Springer Nature Switzerland, 2024, p. 749–760. doi: 10.1007/978-3-031-72069-7_70. [203] B. Demir et al., “MultiGradICON: A Foundation Model for Multimodal Medical Image Registration,” in Biomedical Image Registration, vol. 15249, M. Modat, I. Simpson, Ž. Špiclin, W. Bastiaansen, A. Hering, and T. C. W. Mok, Eds., in Lecture Notes in Computer Science, vol. 15249. , Cham: Springer Nature Switzerland, 2024, p. 3–18. doi: 10.1007/978-3-031-73480-9_1. [204] J. Hu, K. Yu, H. Xian, S. Hu, and X. Wang, “Improving Generalization of Medical Image Registration Foundation Model,” May 10, 2025, arXiv: arXiv:2505.06527. doi: 10.48550/arXiv.2505.06527. [205] X. Song, X. Xu, and P. Yan, “General Purpose Image Encoder DINOv2 for Medical Image Registration,” Feb. 24, 2024, arXiv: arXiv:2402.15687. doi: 10.48550/arXiv.2402.15687. [206] H. Xu et al., “MultiCo3D: Multi-Label Voxel Contrast for One-Shot Incremental Segmentation of 3D Neuroimages,” Mar. 09, 2025, arXiv: arXiv:2503.06598. doi: 10.48550/arXiv.2503.06598. [207] H. Singh, “Generative AI for Synthetic Data Creation: Solving Data Scarcity in Machine Learning,” Available SSRN 5267914, 2025, Accessed: Nov. 22, 2025. [Online]. Available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=52 67914 [208] V. Thambawita et al., “SinGAN-Seg: Synthetic training data generation for medical image segmentation,” PloS One, vol. 17, no. 5, p. e0267976, 2022. [209] L. R. Koetzier et al., “Generating Synthetic Data for Medical Imaging,” Radiology, vol. 312, no. 3, p. e232471, Sept. 2024, doi: 10.1148/radiol.232471. [210] C.-Y. Ung et al., “Artificial Clinic Intelligence (ACI): A Generative AI-Powered Modeling Platform to Optimize Patient Cohort Enrichment and Clinical Trial Optimization,” Cancers, vol. 17, no. 21, p. 3543, 2025. [211] B. D. Killeen et al., “Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling,” Feb. 13, 2025, arXiv: arXiv:2502.09688. doi: 10.48550/arXiv.2502.09688. [212] S. S. Bhuyan et al., “Generative Artificial Intelligence Use in Healthcare: Opportunities for Clinical Excellence and Administrative Efficiency,” J. Med. Syst., vol. 49, no. 1, p. 10, Jan. 2025, doi: 10.1007/s10916-024-02136-1. [213] J. S. Yoon, C. Zhang, H.-I. Suk, J. Guo, and X. Li, “SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation,” in Information Processing in Medical Imaging, vol. 13939, A. Frangi, M. De Bruijne, D. Wassermann, and N. Navab, Eds., in Lecture Notes in Computer Science, vol. 13939. , Cham: Springer Nature Switzerland, 2023, p. 388–400. doi: 10.1007/978-3-031-34048-2_30. [214] L. X. Nguyen, P. S. Aung, H. Q. Le, S.-B. Park, and C. S. Hong, “A new chapter for medical image generation: The stable diffusion method,” in 2023 International Conference on Information Networking (ICOIN), IEEE, 2023, p. 483–486. Accessed: Nov. 22, 2025. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10049010/ [215] N. Konz, Y. Chen, H. Dong, and M. A. Mazurowski, “Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, vol. 15007, M. G. Linguraru, Q. Dou, A. Feragen, S. Giannarou, B. Glocker, K. Lekadir, and J. A. Schnabel, Eds., in Lecture Notes in Computer Science, vol. 15007. , Cham: Springer Nature Switzerland, 2024, p. 88–98. doi: 10.1007/978- 3-031-72104-5_9. 23 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 [216] F. Khader et al., “Denoising diffusion probabilistic models for 3D medical image generation,” Sci. Rep., vol. 13, no. 1, p. 7303, 2023. [217] A. L. Y. Hung et al., “Med-cdiff: Conditional medical image generation with diffusion models,” Bioengineering, vol. 10, no. 11, p. 1258, 2023. [218] J. Wang et al., “Self-improving generative foundation model for synthetic medical image generation and clinical applications,” Nat. Med., vol. 31, no. 2, p. 609– 617, Feb. 2025, doi: 10.1038/s41591-024-03359-y. [219] M. Cobbinah et al., “Diversity in Stable GANs: A Systematic Review of Mode Collapse Mitigation Strategies,” Eng. Rep., vol. 7, no. 6, p. e70209, June 2025, doi: 10.1002/eng2.70209. [220] C. Guo, A. N. Christensen, and M. R. Hannemose, “Med- Art: Diffusion Transformer for 2D Medical Text-to- Image Generation,” June 25, 2025, arXiv: arXiv:2506.20449. doi: 10.48550/arXiv.2506.20449. [221] Y. Dai, Y. Gao, and F. Liu, “Transmed: Transformers advance multi-modal medical image classification,” Diagnostics, vol. 11, no. 8, p. 1384, 2021. [222] Z. Xia, H. Li, and L. Lan, “MedFormer: Hierarchical Medical Vision Transformer with Content-Aware Dual Sparse Selection Attention,” Aug. 05, 2025, arXiv: arXiv:2507.02488. doi: 10.48550/arXiv.2507.02488. [223] P. Chambon et al., “RoentGen: Vision-Language Foundation Model for Chest X-ray Generation,” Nov. 23, 2022, arXiv: arXiv:2211.12737. doi: 10.48550/arXiv.2211.12737. [224] Q. Chen et al., “Towards generalizable tumor synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, p. 11147–11158. Accessed: Oct. 17, 2025. [Online]. Available: http://openaccess.thecvf.com/content/CVPR2024/html/ Chen_Towards_Generalizable_Tumor_Synthesis_CVP R_2024_paper.html [225] L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, p. 3836–3847. Accessed: Nov. 22, 2025. [Online]. Available: http://openaccess.thecvf.com/content/ICCV2023/html/Z hang_Adding_Conditional_Control_to_Text-to- Image_Diffusion_Models_ICCV_2023_paper.html [226] I. E. Hamamci et al., “GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes,” in Computer Vision – ECCV 2024, vol. 15137, A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, and G. Varol, Eds., in Lecture Notes in Computer Science, vol. 15137. , Cham: Springer Nature Switzerland, 2025, p. 126–143. doi: 10.1007/978-3-031-72986-7_8. [227] C. Zhao et al., “MAISI-v2: Accelerated 3D High- Resolution Medical Image Synthesis with Rectified Flow and Region-specific Contrastive Loss,” Aug. 07, 2025, arXiv: arXiv:2508.05772. doi: 10.48550/arXiv.2508.05772. [228] P. Guo et al., “Maisi: Medical ai for synthetic imaging,” in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), IEEE, 2025, p. 4430–4441. Accessed: Oct. 17, 2025. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10943915/ [229] S.-C. Huang, M. Jensen, S. Yeung-Levy, M. P. Lungren, H. Poon, and A. S. Chaudhari, “Multimodal Foundation Models for Medical Imaging-A Systematic Review and Implementation Guidelines,” medRxiv, p. 2024–10, 2024. [230] C. Wu, X. Zhang, Y. Zhang, H. Hui, Y. Wang, and W. Xie, “Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data,” Nat. Commun., vol. 16, no. 1, p. 7866, 2025. [231] S.-C. Huang, M. Jensen, S. Yeung-Levy, M. P. Lungren, H. Poon, and A. S. Chaudhari, “Multimodal Foundation Models for Medical Imaging-A Systematic Review and Implementation Guidelines,” medRxiv, p. 2024–10, 2024. [232] O. Banerjee, H.-Y. Zhou, S. Adithan, S. Kwak, K. Wu, and P. Rajpurkar, “Direct preference optimization for suppressing hallucinated prior exams in radiology report generation,” ArXiv Prepr. ArXiv240606496, 2024, Accessed: Nov. 24, 2025. [Online]. Available: https://arxiv.org/abs/2406.06496 [233] E.-D. M. E. by Direct, “RadQA-DPO: A Radiology Question Answering System with Encoder-Decoder Models Enhanced by Direct Preference Optimization”, Accessed: Nov. 24, 2025. [Online]. Available: https://aclanthology.org/2025.bionlp-1.10v2.pdf [234] S. Pai et al., “Foundation models for quantitative biomarker discovery in cancer imaging,” MedRxiv, 2023, Accessed: Oct. 17, 2025. [Online]. Available: https://pmc.ncbi.nlm.nih.gov/articles/PMC10508804/ [235] A. S. Tejani, T. S. Cook, M. Hussain, T. Sippel Schmidt, and K. P. O’Donnell, “Integrating and Adopting AI in the Radiology Workflow: A Primer for Standards and Integrating the Healthcare Enterprise (IHE) Profiles,” Radiology, vol. 311, no. 3, p. e232653, June 2024, doi: 10.1148/radiol.232653. [236] A. Vian, D. A. Eifer, M. Anes, G. R. Garcia, and M. Recamonde-Mendoza, “Exploring the Feasibility of AI- Assisted Spine MRI Protocol Optimization Using DICOM Image Metadata,” Feb. 04, 2025, arXiv: arXiv:2502.02351. doi: 10.48550/arXiv.2502.02351. [237] L. Melazzini et al., “AI for image quality and patient safety in CT and MRI,” Eur. Radiol. Exp., vol. 9, no. 1, p. 28, Feb. 2025, doi: 10.1186/s41747-025-00562-5. [238] H. R. Tizhoosh, “Foundation models and information retrieval in digital pathology,” in Artificial Intelligence in Pathology, Elsevier, 2025, p. 211–232. Accessed: Oct. 17, 2025. [Online]. Available: https://w.sciencedirect.com/science/article/pii/B9780 32395359700011X [239] K. Clark et al., “The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository,” J. Digit. Imaging, vol. 26, no. 6, p. 1045– 1057, Dec. 2013, doi: 10.1007/s10278-013-9622-7. [240] “MIDRC,” MIDRC. Accessed: Nov. 27, 2025. [Online]. Available: https://w.midrc.org [241] J. Zbontar et al., “fastMRI: An Open Dataset and Benchmarks for Accelerated MRI,” Dec. 11, 2019, 24 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 arXiv: arXiv:1811.08839. doi: 10.48550/arXiv.1811.08839. [242] F. Liu, J. V. Velikina, W. F. Block, R. Kijowski, and A. A. Samsonov, “Fast Realistic MRI Simulations Based on Generalized Multi-Pool Exchange Tissue Model,” IEEE Trans. Med. Imaging, vol. 36, no. 2, p. 527–537, Feb. 2017, doi: 10.1109/TMI.2016.2620961. [243] C. H. McCollough et al., “Low‐dose CT for the detection and classification of metastatic liver lesions: Results of the 2016 Low Dose CT Grand Challenge,” Med. Phys., vol. 44, no. 10, Oct. 2017, doi: 10.1002/mp.12345. [244] “Grand Challenge.” Accessed: Nov. 27, 2025. [Online]. Available: https://w.aapm.org/GrandChallenge/Default.asp [245] X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “ChestX-ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases”. [246] J. Irvin et al., “CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison,” Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, p. 590–597, July 2019, doi: 10.1609/aaai.v33i01.3301590. [247] A. E. W. Johnson et al., “MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports,” Sci. Data, vol. 6, no. 1, p. 317, Dec. 2019, doi: 10.1038/s41597-019-0322-0. [248] K. Yan, X. Wang, L. Lu, and R. M. Summers, “DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning,” J. Med. Imaging, vol. 5, no. 03, p. 1, July 2018, doi: 10.1117/1.JMI.5.3.036501. [249] X. Mei et al., “RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning,” Radiol. Artif. Intell., vol. 4, no. 5, p. e210315, Sept. 2022, doi: 10.1148/ryai.210315. [250] B. H. Menze et al., “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS),” IEEE Trans. Med. Imaging, vol. 34, no. 10, p. 1993–2024, Oct. 2015, doi: 10.1109/TMI.2014.2377694. [251] D. Ouyang et al., “Video-based AI for beat-to-beat assessment of cardiac function,” Nature, vol. 580, no. 7802, p. 252–256, Apr. 2020, doi: 10.1038/s41586-020- 2145-8. [252] F. Prior et al., “The public cancer radiology imaging collections of The Cancer Imaging Archive,” Sci. Data, vol. 4, no. 1, p. 170124, Sept. 2017, doi: 10.1038/sdata.2017.124. [253] W. Bulten et al., “Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge,” Nat. Med., vol. 28, no. 1, p. 154–163, Jan. 2022, doi: 10.1038/s41591-021-01620-2. [254] O. Pelka, S. Koitka, J. Rückert, F. Nensa, and C. M. Friedrich, “Radiology Objects in COntext (ROCO): A Multimodal Image Dataset,” in Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, D. Stoyanov, Z. Taylor, S. Balocco, R. Sznitman, A. Martel, L. Maier-Hein, L. Duong, G. Zahnd, S. Demirci, S. Albarqouni, S.-L. Lee, S. Moriconi, V. Cheplygina, D. Mateus, E. Trucco, E. Granger, and P. Jannin, Eds., Cham: Springer International Publishing, 2018, p. 180–189. [255] S. Subramanian et al., “MedICaT: A Dataset of Medical Images, Captions, and Textual References,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Online: Association for Computational Linguistics, 2020, p. 2112–2120. doi: 10.18653/v1/2020.findings-emnlp.191. [256] J. Yang et al., “MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification,” Sci. Data, vol. 10, no. 1, p. 41, Jan. 2023, doi: 10.1038/s41597-022-01721-8. [257] M. Antonelli et al., “The Medical Segmentation Decathlon,” Nat. Commun., vol. 13, no. 1, p. 4128, July 2022, doi: 10.1038/s41467-022-30695-9. [258] R. Souza et al., “An open, multi-vendor, multi-field- strength brain MR dataset and analysis of publicly available skull stripping methods agreement.,” NeuroImage, vol. 170, p. 482–494, Apr. 2018, doi: 10.1016/j.neuroimage.2017.08.021. [259] S. G. Armato et al., “The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans,” Med. Phys., vol. 38, no. 2, p. 915–931, Feb. 2011, doi: 10.1118/1.3528204. [260] T. J. Littlejohns et al., “The UK Biobank imaging enhancement of 100,000 participants: rationale, data collection, management and future directions,” Nat. Commun., vol. 11, no. 1, p. 2624, May 2020, doi: 10.1038/s41467-020-15948-9. [261] The National Lung Screening Trial Research Team, “Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening,” N. Engl. J. Med., vol. 365, no. 5, p. 395–409, Aug. 2011, doi: 10.1056/NEJMoa1102873. [262] D. S. Marcus, T. H. Wang, J. Parker, J. G. Csernansky, J. C. Morris, and R. L. Buckner, “Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults,” J. Cogn. Neurosci., vol. 19, no. 9, p. 1498–1507, Sept. 2007, doi: 10.1162/jocn.2007.19.9.1498. [263] D. C. Van Essen, S. M. Smith, D. M. Barch, T. E. J. Behrens, E. Yacoub, and K. Ugurbil, “The WU-Minn Human Connectome Project: An overview,” Mapp. Connect., vol. 80, p. 62–79, Oct. 2013, doi: 10.1016/j.neuroimage.2013.05.041. [264] A. L. Goldberger et al., “PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals,” Circulation, vol. 101, no. 23, June 2000, doi: 10.1161/01.CIR.101.23.e215. [265] “Alexandr Wang on AI’s Potential and Its ‘Deficiencies’ | TIME.” Accessed: Aug. 31, 2025. [Online]. Available: https://time.com/7296215/alexandr-wang- interview/?utm_source=chatgpt.com 25 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 [266] W. Yang, H. Zhang, W. Tan, Y. Sun, and B. Yan, “A Self-Supervised Paradigm for Data-Efficient Medical Foundation Model Pre-training: V-information Optimization Framework,” Apr. 06, 2025, arXiv: arXiv:2408.07107. doi: 10.48550/arXiv.2408.07107. [267] I. Dayan et al., “Federated learning for predicting clinical outcomes in patients with COVID-19,” Nat. Med., vol. 27, no. 10, p. 1735–1743, Oct. 2021, doi: 10.1038/s41591-021-01506-3. [268] X. Zhang, H. Deng, R. Wu, J. Ren, and Y. Ren, “PQSF: post-quantum secure privacy-preserving federated learning,” Sci. Rep., vol. 14, no. 1, p. 23553, Oct. 2024, doi: 10.1038/s41598-024-74377-6. [269] Y. Shi, W. Xia, C. Niu, C. Wiedeman, and G. Wang, “Enabling Competitive Performance of Medical Imaging with Diffusion Model-generated Images without Privacy Leakage,” Feb. 15, 2024, arXiv: arXiv:2301.06604. doi: 10.48550/arXiv.2301.06604. [270] K. Huang et al., “A foundation model for clinician- centered drug repurposing,” Nat. Med., vol. 30, no. 12, p. 3601–3613, Dec. 2024, doi: 10.1038/s41591-024- 03233-x. [271] Z. Wang, E. Khatibi, and A. M. Rahmani, “MedCoT- RAG: Causal Chain-of-Thought RAG for Medical Question Answering,” Aug. 20, 2025, arXiv: arXiv:2508.15849. doi: 10.48550/arXiv.2508.15849. [272] N. Fathi, T. Scholak, and P.-A. Noël, “Unifying Autoregressive and Diffusion-Based Sequence Generation,” Apr. 08, 2025, arXiv: arXiv:2504.06416. doi: 10.48550/arXiv.2504.06416. [273] J. Hu et al., “ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformer,” Mar. 13, 2025, arXiv: arXiv:2412.07720. doi: 10.48550/arXiv.2412.07720. [274] S. Azizi, S. Kundu, M. E. Sadeghi, and M. Pedram, “MambaExtend: A Training-Free Approach to Improve Long Context Extension of Mamba,” presented at the The Thirteenth International Conference on Learning Representations, Oct. 2024. Accessed: Sept. 01, 2025. [Online]. Available: https://openreview.net/forum?id=LgzRo1RpLS&utm_s ource=chatgpt.com [275] “On the Promise for Assurance of Differentiable Neurosymbolic Reasoning Paradigms.” Accessed: Sept. 01, 2025. [Online]. Available: https://arxiv.org/html/2502.08932v1?utm_source=chatg pt.com [276] G. Wang and F.-L. Fan, “Dimensionality and dynamics for next-generation artificial neural networks,” Patterns, vol. 6, no. 8, Aug. 2025, doi: 10.1016/j.patter.2025.101231. [277] S. Sinha, P. Gehler, F. Locatello, and B. Schiele, “TeST: Test-time Self-Training under Distribution Shift,” Sept. 23, 2022, arXiv: arXiv:2209.11459. doi: 10.48550/arXiv.2209.11459. [278] DeepSeek-AI et al., “DeepSeek-V3 Technical Report,” Feb. 18, 2025, arXiv: arXiv:2412.19437. doi: 10.48550/arXiv.2412.19437. [279] J. Liang, R. He, and T. Tan, “A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts,” Int. J. Comput. Vis., vol. 133, no. 1, p. 31–64, Jan. 2025, doi: 10.1007/s11263-024-02181-w. [280] M. Moor et al., “Foundation models for generalist medical artificial intelligence,” Nature, vol. 616, no. 7956, p. 259–265, Apr. 2023, doi: 10.1038/s41586-023- 05881-4. [281] H. Chao et al., “Deep learning predicts cardiovascular disease risks from lung cancer screening low dose computed tomography,” Nat. Commun., vol. 12, no. 1, p. 2963, May 2021, doi: 10.1038/s41467-021-23235-4. [282] C. Niu et al., “Medical multimodal multitask foundation model for lung cancer screening,” Nat. Commun., vol. 16, no. 1, p. 1523, Feb. 2025, doi: 10.1038/s41467-025- 56822-w. [283] R. Harrison, “Alpha + Beta hardware (fall 2025),” Empire AI. Accessed: Sept. 01, 2025. [Online]. Available: https://empireai.freshdesk.com/support/solutions/articles /157000363466-alpha-beta-hardware-fall-2025- [284] “A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future,” ar5iv. Accessed: Sept. 01, 2025. [Online]. Available: https://ar5iv.labs.arxiv.org/html/2309.15402 [285] L. Jiao et al., “Causal Inference Meets Deep Learning: A Comprehensive Survey,” Res. Wash. DC, vol. 7, p. 0467, 2024, doi: 10.34133/research.0467. [286] X. Yu et al., “Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning,” July 26, 2025, arXiv: arXiv:2506.09853. doi: 10.48550/arXiv.2506.09853. [287] K. Alhamoud et al., “FedMedICL: Towards Holistic Evaluation of Distribution Shifts in Federated Medical Imaging,” July 11, 2024, arXiv: arXiv:2407.08822. doi: 10.48550/arXiv.2407.08822. [288] H. Guan, P.-T. Yap, A. Bozoki, and M. Liu, “Federated learning for medical image analysis: A survey,” Pattern Recognit., vol. 151, p. 110424, July 2024, doi: 10.1016/j.patcog.2024.110424. [289] G. Papadopoulos, P. J. Edwards, and A. F. Murray, “Confidence estimation methods for neural networks: a practical comparison,” IEEE Trans. Neural Netw., vol. 12, no. 6, p. 1278–1287, Nov. 2001, doi: 10.1109/72.963764. [290] Z. Shi et al., “Generate-then-Ground in Retrieval- Augmented Generation for Multi-hop Question Answering,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand: Association for Computational Linguistics, 2024, p. 7339–7353. doi: 10.18653/v1/2024.acl-long.397.

Instant research discovery