Paper deep dive
BioMed-Agent-RL: A Meta Learning, All You Need for Biomedical Applications
Md Asaduzzaman Jabin, Zihao Wu, Tianming Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/25/2026, 7:32:26 AM
Summary
The paper introduces BioMed-Agent-RL, a unified medical agent framework designed to improve clinical reasoning and diagnostics by addressing issues like lesion noise, modality misalignment, and hallucinations in Clinical Vision Large Language Models (C-VLLMs). The system employs a multimodal meta-learning approach with adaptive orchestration, utilizing Reinforcement Learning (RL) techniques including Clinical Context-aware Preference Optimization (CPO), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO) with dynamic entropy regulation. The agent integrates expert models for lesion segmentation and reasoning, aggregating their outputs to produce reliable clinical insights. Experimental results across multiple benchmarks (RAD-VQA, SLAKE, Path-VQA, MIMIC-CXR, IU-Xray) show that the proposed C-GRPO method significantly outperforms baselines like GPT-5, LLaVA-Med, and DeepSeek-VL, achieving up to ~73% accuracy.
Entities (19)
Relation Signals (15)
Md Asaduzzaman Jabin β affiliatedwith β The University of Georgia
confidence 95% Β· Md Asaduzzaman Jabin, Zihao Wu, and Tianming Liu are with The University of Georgia
Tianming Liu β affiliatedwith β The University of Georgia
confidence 95% Β· Md Asaduzzaman Jabin, Zihao Wu, and Tianming Liu are with The University of Georgia
Zihao Wu β affiliatedwith β The University of Georgia
confidence 95% Β· Md Asaduzzaman Jabin, Zihao Wu, and Tianming Liu are with The University of Georgia
BioMed-Agent-RL β evaluatedon β RAD-VQA
confidence 95% Β· we train and fine-tune our agent on 5 medical datasets ... RAD-VQA
BioMed-Agent-RL β evaluatedon β Path-VQA
confidence 95% Β· we train and fine-tune our agent on 5 medical datasets ... Path-VQA
BioMed-Agent-RL β evaluatedon β SLAKE
confidence 95% Β· we train and fine-tune our agent on 5 medical datasets ... SLAKE
BioMed-Agent-RL β evaluatedon β MIMIC-CXR
confidence 95% Β· we train and fine-tune our agent on 5 medical datasets ... MIMIC-CXR
BioMed-Agent-RL β evaluatedon β IU-XRay
confidence 95% Β· we train and fine-tune our agent on 5 medical datasets ... IU-Xray
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The current progress of Clinical Vision Large Language Models (C-VLLMs) has substantially improved digital diagnostics, still these frameworks often endure lesion noises, modality misalignment, hallucination, and missed contextual grounding in complex clinical cases. Moreover, prevailing agent systems usually depend on static and non-adaptable pipelines and lack the versatility necessary for complex medical reasoning. To resolve these difficulties, we present BioMed-Agent-RL, a unified medical agent that incorporates adaptive orchestration, policy, and reward-based reinforcement learning (RL) models for biomedical applications. To ensure reliability, it invokes clinical context-aware preference optimization (CPO), direct preference optimization (DPO), and group relative policy optimization (GRPO) with dynamic entropy regulation. This pipeline utilizes a multimodal meta-learning approach that operates as a field-specific expert and human judgment synthesizer. The agent adaptively utilizes a set of model-level expertise, such as clinical grounding and reasoner, lesion segmenter, and field-specific synthesizer, across various clinical modalities (e.g., X-ray) by utilizing an iterative and adaptive RL approach. The agent learns to seriously synthesize misleading, conflicting vision cues and trust in inherent reasoning, while specialist advice is faulty. An intensive ablation study is conducted across multiple benchmarks, and the agent significantly outperforms existing state of the art models, such as GPT-5, attaining up to ~73% accuracy (gain of ~5%) over contemporary baselines. As a result, the framework suggests a new standard for building factual, reliable, robust, and expert-like intelligent agent systems for independent clinical reasoning.
Tags
Links
- Source: https://arxiv.org/abs/2608.21864v1
- Canonical: https://arxiv.org/abs/2608.21864v1
Trouble viewing inline? Open PDF directly β
Full Text
35,410 characters extracted from source content.
Expand or collapse full text
IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. X, NOVEMBER 2020 1 ο BioMed-Agent-RL: A Meta Learning, All You Need for Biomedical Applications Md Asaduzzaman Jabin, Member, IEEE, Zihao Wu, and Tianming Liu, Member, IEEE AbstractβThe current progress of Clinical Vision Large Language Models (C-VLLMs) has substantially improved digital diagnostics, still these frameworks often endure lesion noises, modality misalignment, hallucination, and missed contextual grounding in complex clinical cases. Moreover, prevailing agent systems usually depend on static and non-adaptable pipelines and lack the versatility necessary for complex medical reasoning. To resolve these difficulties, we present BioMed-Agent-RL, a unified medical agent that incorporates adaptive orchestration, policy, and reward-based reinforcement learning (RL) models for biomedical applications. To ensure reliability, it invokes clinical context-aware preference optimization (CPO), direct preference optimization (DPO), and group relative policy optimization (GRPO) with dynamic entropy regulation. This pipeline utilizes a multimodal meta- learning approach that operates as a field-specific expert and human judgment synthesizer. The agent adaptively utilizes a set of model-level expertise, such as clinical grounding and reasoner, lesion segmenter, and field-specific synthesizer, across various clinical modalities (e.g., X-ray) by utilizing an iterative and adaptive RL approach. The agent learns to seriously synthesize misleading, conflicting vision cues and trust in inherent reasoning, while specialist advice is faulty. An intensive ablation study is conducted across multiple benchmarks, and the agent significantly outperforms existing state of the art models, such as GPT-5, attaining up to ~73% accuracy (gain of ~5%) over contemporary baselines. As a result, the framework suggests a new standard for building factual, reliable, robust, and expert-like intelligent agent systems for independent clinical reasoning. Index TermsβBiomedical Agent, Contextual-aware Preference Optimization (CPO), Direct Preference Optimization (DPO), Group Relative Policy Optimization (GRPO), and Reinforcement Learning (RL). I. INTRODUCTION HE recent progress of multimodal vision large language models (VLLMs) into biomedical Md Asaduzzaman Jabin, Zihao Wu, and Tianming Liu are with The University of Georgia, Athens, Georgia, GA 30605 USA. (e-mail: mj71006, zihao.wu1, tliu@uga.edu). applications [1] across various clinical modalities like MRI, CT, X-ray, and Gross, etc., has exhibited substantial possibilities for intelligent reasoning and clinical diagnostics. However, VLLMs trained on large multimodal clinical data depict promising performance to some extent, yet a single backbone often fails in reasoning [2] of various clinical subfields (e.g., radiology). Still, the advancement of clinical intelligence systems depicts several major challenges [3]- high computation cost associated with human- feedback alignment and optimization, the limited generalization capabilities of sole backbone across distinct modalities, hallucinations caused by multimodal misalignment, and non-adaptive, non-iterative essence of contemporary agent systems. Achieving specific and most accurate iterative reasoning direction in foundation models generally entails reinforcement learning with human feedback (RLHF) with proximal policy optimization (PPO) [4], a method that is often computation-intensive and time consuming due to need of huge manual data annotation and tuning a huge distinct reward model. To tackle this, direct preference optimization (DPO) [5] has surfaced as firm and lightweight option which optimizes LLM models by drawing policy function in the loop through binary cross-entropy, bypass external reward model. While DPO elevates general contextual conversations, but expert-level clinical tasks necessitate further greater precision. In the medical field, a generalist LLM backbone frequently struggles [6] to tackle the reasoning and lesion grounding task necessary for diverse branches like pathology, radiology, and surgery, etc. This requires an agentic framework, multimodal clinical agent, able to adaptively orchestrating specific mechanism [7] for multimodal grounding, lesion reasoning and segmentation, report and summary generation across various clinical cues like, MRI, CT, and X-rays. In spite of the triumph of such agents, they repeatedly endure from multimodal misalignment where the model mostly T 2 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 prefers text-based knowledge over visual embedding. Therefore, it leads to erroneous reasoning and fatal hallucinations. To alleviate these issues, clinical context preference optimization (CPO) [8] selects excellent- quality preference cues by presenting βrejectedβ instances through region-specific lesion noising. However, the noising often disturbs visual reasoning of vital investigative areas and causes the inclusion of hallucinations [9]. However, by applying weights over these cues through agent pipelines, the model grasps to prefer critical visual embedding over hallucinated words. To address the problems with solely generalist backbone, a multi-agent system [10] can be a great solution. But, their architecture commonly depend on non-adaptive, static and rigid frameworks. Finally, to surmount problems of multi-agent pipelines, a group relative policy optimization (GRPO) framework [11], [12] can be enhanced with intra-group relative reward model, avoiding the necessity of a critic model. To optimize co-operation between the expert model and assistant model, curriculum guided reinforcement learning (C-RL) [13] with GRPO can be a viable solution. Gradually, coaching the model through the exploration and exploitation dilemma on difficult cases, which leads to hallucinations. In this manuscript, we propose an integrated clinical agent that relies on LLM model orchestration with curriculum-based reinforcement learning (RL). Our key contributions are, (a) A multimodal alignment architecture with reasoning, lesion segmentation module and use a weighted DPO framework guided by diagnostic relevance ranking; (b) Implementing a clinical-aware preference optimization (CPO) framework with local lesion-noising to improve visual and textual alignment; and (c) A agent collaboration with GRPO based curriculum learning (C-GRPO) to surpass prominent generalist model, GPT-5 [14] in Fig. 2. The performance evaluation between DPO, CPO and C-GRPO- (a) architecture-specific; and (b) Dataset-specific performance efficiency gains of C-GRPO RL approach. Fig. 1. The overview of BioMed-Agent-RL- (a) The reasoning loop; and (b) The agent comparison paradigm- (i) Single backbone agent, (i) Multisystem agent, and (i) C-GRPO BioMed-Agent-RL (Ours). 3 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 complex clinical and grounding across various metrics and benchmarks. The more details about agent backbone paradigm is depicted in Figure 1(b). The model acquires the ability to wisely combine contradictory recommendations and trust in intrinsic reasoning, while the expert entity collapses. This integrated architecture founds a new standard for automated intelligence systems. I. PRIOR WORKS A. Clinical AI Agent Recently, AI agent has emerged with great prospects in clinical applications and reasoning. Some are unimodal, and some are multimodal in nature. AI agent expertise in medical setups [15], [16], [17], such as clinical, radiology, pathology, radiotherapy, dermatology, dental, genomics, EHR, and surgical have gained commendable recognition in the research arena. But most of the agents are either unimodal (text-based) or perform better in textual reasoning. Moreover, they are prone to multimodal misalignment, unstable and computation- intensive, limited generalization, a static sole backbone with no self-evolving, no adaptation, and no re-correction mechanism. As a result, they underperform by causing factual misalignment, false contextual reasoning, inefficient lesion grounding and segmentation, and hallucination [3]. B. Reinforcement Learning Based Agent On the other hand, reinforcement learning (RL) is used as a great tool in the agentic AI space, due to its self- adaptation, iterative, and self-evolving mechanisms. RL concepts [18] like, DPO, PPO, CPO, and GRPO, etc. are used in generalist tasks like, dialogue continuation, report generation, captioning, self-reasoning, spatial recognition, and vision cue generation related to medical fields. But, they often suffer inaccurate processes with human feedback alignment, instability, a generalist backbone, and in-adaptively orchestrate LLM tools, non- iterative and mismatched co-operation between expert tools [19] (refer to Figure 1(b)). TABLE I THE PERFORMANCE COMPARISON OF RL APPROACHES ACROSS 5 CLINICAL BENCHMARKS. Metrics and Checkpoints Backbone RL Methods RAD- VQA SLAK E PATH- VQA MIMIC- CXR IU-Xray LLaVA-Med DPO 47.54 41.65 40.29 47.37 37.42 CPO 50.21 45.87 45.49 47.59 43.93 C-GRPO 50.43 51.31 49.59 51.9 49.95 DeepSeek-VL DPO 56.2 61.12 51.71 49.94 49.79 CPO 61.19 65.12 55.73 55.56 55.52 C-GRPO 60.77 62.65 60.45 60.89 60.91 GPT-5 DPO 60.26 57.1 52.36 50.51 55.18 CPO 64.6 61.45 57.34 55.8 60.46 C-GRPO 63.23 65.19 61.09 63.22 64.02 Ours DPO 65.29 60.43 56.8 61.33 60.77 CPO 68.72 65.9 63.43 67.79 66.61 C-GRPO 73.08 71.71 70.68 72.55 72.68 4 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 I. METHODS In this part, we provide a concise overview of VLLMs, multimodal agent collaboration, DPO, CPO, and GRPO techniques with dynamic entropy regulation. VLLM: Let multimodal clinical input x in = (x vis , x t ), where, visual input is x vis and corresponding text input is x t . They auto- iteratively guess the following tokenβs distribution to generate clinical reasoning and grounding response y out . Multi-Agent System: To help with complicated agent pipelines, multi agent clinical systems harmonize various expert agents. Our scenario imitates a clinical setup for radiology diagnostics: Expert Agent (EA) ο Clinical Specialist ο Expert Model (EA). If an expert agent a i β A N observe an policy method ν ν ν (ν₯ νν | ν¦ νν’ν‘ ) for a multimodal vision cue x in = (x vis , x t ), the multimodal agent systems would be, ν΄ νΈν΄ ν =ν νΈν΄ 1 ,ν νΈν΄ 2 ,ν νΈν΄ 3 ,,,,,,,ν νΈν΄ ν . Where, N = total number of agent and ν ν ν is the policy model. The pipeline progresses as follows: (a) Task captioning: ν νΈν΄ ννν agent selects the category of task based on the task caption and category of tasks via ranking scores, ν ν = ννν ννν₯ ν ν ν νΈν΄ ννν ( ν | ν₯ νν ); and (b) Expert model response: Every individual captioning agent ν νΈν΄ ν ν after ranking most-k input sequence, and select the corresponding expert agent from the pool, thus produce responses, ν¦ ν ν ~ ν ν νΈν΄ ν ν (ν¦ νν’ν‘ | ν₯ ν£νν ); and (c) Aggregation agent: The output from expert agent, then fend into aggregator (ν νΈν΄ νν ) to choose and finalize the appropriate output, ν¦ ννννν ~ ν ν νΈν΄ νν (ν¦ νν’ν‘ | ν₯ νν ,ν¦ ν ν ). Direct Preference Optimization (DPO): Preference optimization has demonstrated to be very effective in fine-tuning language models (LMs) and leads to substantial coordination between target and model behavior. If a DPO [5] policy method ν ν ν·νν produces a contingent distribution ν ν ν·νν ( ν¦ νν’ν‘ | ν₯ νν ) across a preference data ν·=ν₯ νν ,ν¦ νννν ,ν¦ ννννν , where ν₯ νν , ν¦ νννν , and ν¦ ννννν represent the multimodal input, preferred response, and disliked output, respectively. The likelihood score of choosing ν¦ νννν is designed as, ν ( ν¦ νννν | ν¦ ννννν )= ν (ν ( ν₯ νν ,ν¦ νννν ) βν(ν₯ νν ,ν¦ ννννν ). Where, ν (.) = the sigmoid non-linear function. The DPO optimization can be calculated through loss function over preference data D, β ν·νν (ν ν ν·νν ,ν ν νννν )= βνΈ ( ν₯ νν , ν¦ νννν , ν¦ ννννν ) ~ ν· [logν ( νΌlog ν ν ν·νν ( ν¦ νννν | ν₯ νν ) ν ν νννν ( ν¦ νννν | ν₯ νν ) ) β νΌlog ν ν ν·νν ( ν¦ ννννν | ν₯ νν ) ν ν νννν ( ν¦ ννννν | ν₯ νν ) ] Here, ν ν νννν = preference policy. Context-aware Preference Optimization (CPO): During DPO, the likelihood score (S) is used as a penalty representing the influence of individual preference-pair data on the total fine-tuning process. To avoid underfitting caused by a small value from the DPO loss function, normalization is applied to achieve better context-aware preference. It maps every score within a fixed range with its variance and mean intact. The normalized clinical relevance score would be, ν Μ = νβ ν ν . Where, (ν,ν) are the mean and variance pair. Then the score is trimmed into [νΌ,ν½]. Here, νΌ,ν½ are chosen as lower and upper bound respectively. So the modified weighted loss function with diagnostic relevance can be computed as, β νΆνν (ν ν νΆνν ,ν ν νννν )= βνΈ (ν₯ νν , ν₯ β , ν¦ νννν , ν¦ ννννν , ν Μ ) ~ ν· [ ν Μ logν(νΌlog ν ν ν·νν (ν¦ νννν | ν₯ νν ) ν ν νννν (ν¦ νννν | ν₯ νν ) )β νΌlog ν ν ν·νν ( ν¦ ννννν | ν₯ β ) ν ν νννν ( ν¦ ννννν | ν₯ β ) ] Group Relative Policy Optimization (GRPO): The GRPO method [11] escapes fine-tuning a critic function by utilizing within-group relative rewards to enhance the policy function. For each multimodal inquiry x in , the pipeline extracts N output responses ν¦ 1 ,ν¦ 2 ,,,,,,ν¦ ν , which are rated to obtain a list of rewards ν 1 ,ν 2 ,,,,,,ν ν . It always chooses and contains normalized leverage points and refreshes the policy function with a trimmed objective function similar PPO [4]. The GRPO optimization would be, β νΊν νν ( ν ) = νΈ ( ν₯ νν , ν¦ ν ) [ 1 ν β( min(ν ν ν΄ ν ,νννν ( ν ν ,1βνΌ,1+νΌ ) ν΄ ν ) β ν ν ν½ ν· νΎνΏ ( ν ν νΊν νν | ν ν νννν )]. Here, each reward component would be, ν ν = ν ν νΊν νν ( ν¦ ν | ν₯ νν ) ν ν ννν ( ν¦ ν | ν₯ νν ) , ν ν ννν = old GRPO policy function, and νΌ,ν½ are the agentβs hyper-parameters, respectively. So, the final agent advantage component would be, ν΄ ν = ν ν βν(ν ν | ν=0 ν ) ν(ν ν | ν=0 ν ) . The GRPO enhance policy based learning through grouped relative policy rewards rather than a critic function. Curriculum Design (C-GRPO) and Dynamic Entropy Regulation: We classify jobs based on the performance of an expertβs assessment, ν= ν΄νΆ (ν¦ ν ν ,ν¦ β ). The whole dataset can be split into 3 complexity levels: totally accurate expert outputs (S=1, tagged as easy (D Easy )), partially accurate results (0< S < 1, marked as medium (D Medium )), and entirely incorrect findings (S = 0, designated as hard (D Hard )). So, the full dataset would be, ν·= ν· νΈνν ν¦ βͺ ν· ννννν’ν βͺ ν· ν»ννν . Therefore, we create a triad-step curriculum based on a GRPO approach to enhance expert knowledge, like when to apply expert knowledge precisely and when to depend on their individual perception to resolve multimodal cues. 5 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 However, we apply GRPO [11] as our base RL curriculum approach. For each multimodal query x, aggregation policy ν ν νΈν΄ νν producing a collection of N responses ν¦ ννννν ν ν=1 ν . Each reward component ν ν for each output is calculated by a reward template ν νννννν‘ β 0, 0.5 and reward accuracy ν ννν’νννν¦ β 0, 1, and computes the conditional advantage component ν΄ ν for policy model update. To attain dynamic entropy control, we present an entropy regularization into our GRPO optimization function, β νΆβνΊν νν ( ν ) = νΈ [β νΊν νν ( ν ) + νΎ ν ν» ν‘ (ν ν νΈν΄ νν )]--(1) Where, ν» ν‘ = β β ν ν‘,ν .νννν ν‘,ν ν ν=1 . And, ν ν‘ = νννν‘ννν₯ ( ν ν‘ ν ), T = vocabulary token size, softmax logits ν ν‘ β ν ν , ν = temperature, νΎ ν = entropy penalty co-efficient is adaptively established based on curriculum level S of current cues. To mention, νΎ ννν ν¦ ~ 0, νΎ ννννν’ν > 0, and νΎ βννν ~ 1 (>> νΎ ννννν’ν ) respectively for easy (S = 1), medium (0<s<1), and hard (S =0) cues. In case of hard cues, a very strong positive penalty is implied vigorously to stimulate exploration, forcing the agent to deviate from deceptive expert suggestions. Initial Captioning Expert: Contemporary task- specific agent approaches like Agent Hospital [20], depend on preset or rule-based allocation plans and do not directly improve the captioning mechanism. Therefore, expert-based routing is fixed and misses flexible optimization over the clinical data. To handle those drawbacks, we formulate our initial task captioning module as a tunable policy ν ν νΈν΄ ννν that allocates every instance to one of K domain experts, where K = 7 for our BioMed-Agent-RL. Supervising is obtained from method-dependent ground-truth offered by dataset (e.g., radiology image ο radiologist LLM backbone), allowing methodical positioning between input and medical expertise. Then, the chosen caption can be denoted based on ranking, tasks via ranking scores, ν ν = ννν ννν₯ ν ν ν νΈν΄ ννν ( ν | ν₯ νν ), where x = multimodal clinical cue. We improve this strategy utilizing GRPO with a combined reward model comprising of template restrictions for report generation ( ν νννννν‘ / νννν‘ννν β 0.0,0.50 ) and reward accuracy ( ν ννν’νννν¦/ νΉ1 β 0.0, 1.0 ) . This kind of reward model promotes assigning of both precise specialty and systematic semantic reasoning results. Role Specialist: After choosing appropriate captions, we employ various robust pipelines as field specialists ν νΈν΄ ν ν to facilitate moderately precise evaluations. This assists following reference by a human expert. In our framework, we utilize outputs from K specialists as pointers for every input sample. We collect the human expert opinions to autonomously incorporate in the data and backbone. So expert output would be, ν¦ ν ν ~ ν ν νΈν΄ ν ν (ν¦ νν’ν‘ | ν₯ νν ). This guarantees framework efficacy and bypasses dominant polling, which could dominate subordinate views and assign the task for final call by a physician. Aggregation Agent: After expert-tier reasoning is produced, the ultimate investigative judgement could be taken by a medical expert who combines diverse specialist views to the final output. In this integration stage, both human and agent expert outputs may vary in terms of reliability or, disagree with one another. However, majority voting may result in depriving Fig. 3. The query category and complexity level analysis for our C-GRPO based BioMed- Agent-RL (1200 instances per category, and 2000 data points per complexity). 6 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 minority explanations may cause propagation of structured errors. Therefore, we are proposing adaptive aggregation policy with entropy regulation, [21] policy- based to reveals attending physician in complex clinical cases, thus empowering systemic adaptation. This policy C-GRPO entropy controls the exploitation and exploration balance. To add more, low entropy fosters agreement-based exploitation while expert consensus is credible. However, high entropy promotes exploratory thinking when unpredictability exists. By incorporating entropy regulation to RL pipeline, C-GRPO reduces error buildup, imbalance between LLM experts, and improves consistency in final outputs. IV. EXPERIMENTS A. Datasets and Baselines We measure BioMed-Agent-RL over 2 agent TABLE I THE QUERY COMPLEXITY ANALYSIS FOR C-GRPO BIOMED-AGENT-RL (6000 INSTANCES). Reasoning difficulty in various dimensions Query Categories Query Complexity RAD- VQA SLAKE PATH- VQA MIMIC- CXR IU-Xray Close-ended Easy 73.08 72.98 72.45 73.01 72.88 Medium 68.83 68.55 68.69 68.59 68.67 Hard 61.53 61.71 62.18 62.66 62.93 Open-ended Easy 65.21 65.48 64.64 64.7 65.35 Medium 59.67 58.11 57.83 58.03 57.97 Hard 53.56 52.89 52.78 51.99 51.64 Grounding-based Easy 70.94 71.79 71.86 71.91 71.56 Medium 66.82 65.95 65.53 65.83 65.62 Hard 64.22 63.49 63.11 63.82 63.74 Attribution- based Easy 71.31 71.73 70.99 71.39 71.45 Medium 67.51 67.9 66.79 67.12 67.26 Hard 63.29 63.82 63.4 63.92 64.03 Summary-based Easy 72.97 72.41 72.65 72.83 72.48 Medium 67.32 68.79 68.89 68.95 68.33 Hard 63.78 62.99 63.55 63.09 63.68 Fig. 4. The BioMed-Agent-RL qualitative result comparison for different RL Algorithms- (a) VQA task, and (b) Report generation task. 7 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 architectural setups: (a) Single Backbone, and (b) Domain-specific list of VLLMs experts. Particularly, we incorporate the performance evaluation of our proposed agent with LLaVA-Med [22], DeepSeek-VL [23], and GPT-5 [13]. We train and fine-tune our agent on 5 medical datasets upon 2 categories: (a) VQA domain- RAD-VQA [24], SLAKE [25]; and (b) Report generation- Path-VQA [26], MIMIC-CXR [27], and IU-Xray [28]. The evaluation query types are a mixture of multimodal closed-ended (e.g., yes/ no), open-ended (e.g., abnormality type), grounding-based (e.g., counting), attribution-based (e.g., shape, size, and density), Q&A, and summary generation tasks. We rigorously follow the standard for splitting the datasets into 40%: 30%: 30% for train, validation, and test sets, respectively. To promote a curriculum learning approach, we additionally grouped train and validation sets into prior categories- VQA domain (dialogue) and Report generation (explanation). And, each sub-categories further sampled into 3 complexity levels, easy, medium, and hard, based on the subject-expertβs feedback. And, for DPO and CPO optimization, each data point contains preferred and non- preferred data fields. Overall, our experiment cases are combinations of 3 RL methods, 4 model backbones, 2 benchmark categories (5 datasets), 5 query categories, and 3 query complexity levels. In total, 3 x 4 x 5 x 5 x 3 = 900 experimental cases are examined retrospectively on the test data to illustrate the agentβs robustness against specialist ground truth and confirm no potential data leak. B. Implementations We employ MMedAgent [10] as a backbone and develop the prompt blueprint to clearly designate mandatory inference structure based on VQA and report generation query category. It involves using <thinking> and <gpt_answer> tag to individually enclose reasoning and lesion grounding process, and inference answer. More details about the workflow is depicted in Figure 1 (a). A curriculum based GRPO algorithm (C-GRPO) is implemented to make the agentic system more robust, accurate grounding, and less hallucinated results (refer to Algorithm 1). The agentβs hyperparameters are- batch size- 256, rollouts- 8, temperature- 9.0, learning rate- 1x10 -5 , KL curriculum coefficients are- 3x10 -3 , 5x10 -5 , 7x10 -7 , # of specialists- 3, and dynamic entropy coefficients- 0.03, 0.003, 0.0003 respectively for query complexity (hard (s = 0), medium (0<s<1), and easy (s=1) multimodal cues). The baseline training and tuning are done on Nvidia 8xA5000, 192 GB VRAM. The whole process took 10~12 days straight to get the final agent checkpoints and inference results. C. Ablation Studies In this part, we assess the overall performance of BioMed-Agent-RL, striving to solve subsequent challenges: (a) Does C-GRPO enhance comprehensive multimodal reasoning and grounding?, (b) Are the performance improvements robust across all datasets?, (c) Is the performance enhancement model architecture independent?, (d) does C-GRPO helped in all query categories and complexity levels?. From Figure 2(a), the metrics of average F1 (%) across 3 RL algorithm at 4 diverse backbone, and combined over 5 clinical benchmarks. Throughout all architectures, C- GRPO uniformly attains better efficiency contrasted to CPO and DPO algorithms. The mean F1 (%) ranges from 49.95% for LLaVA-Med to 73.08% for C-GRPO based MMedAgent checkpoints (refer to yellow bar). The uncertainty bars (error bar window) indicate 95% intervals of confidence is figured across 5 benchmarks, showing steady performance increases. These performance outcomes denotes that C-GRPO offers framework-neutral improvements in mixed modality grounding and reasoning. On the other hand, Figure 2(b) exhibits dataset-specific efficiency gains of C-GRPO proportional to CPO, and DPO RL approaches. Involving all 5 biomedical benchmarks, C-GRPO generates steady affirmative improvements with average gains varying from 1% to 10.5% of βνΉ1 (%). During each iteration of RAD-VQA benchmarks, C-GRPO running performance improved from 2.9% to ~4% (blue error bars) while fine- tuning for the IU-Xray dataset. Similarly, while 3 rd , 4 th and 5 th benchmark, the agent is consistently learning and improving grain F1(%) of ~5%, 4.8% and ~6.5% respectively. It can be noticed that the gain from DPO to C-GRPO (C-GRPO-DPO, refer to red curve) shows much higher than the CPO to C-GRPO (C-GRPO β CPO) curves, which denotes the actual precedence of performance in descending order, C-GRPO > CPO > DPO. More details about performance are shown in Table 1, while the best performance (F1 (%)) for each benchmark and backbone is marked with green text. It can be claimed that with the consistent fine-tuning of each benchmark, the modelβs reasoning, grounding, and generalization improved with steady upward non-linear trends. Figure 3 examines our C-GRPO based BioMed- Agent-RL output across 5 query categories and 3 query complexities. As displayed in Figure 3(a), efficiency persists comparatively firm across query categories- close-ended, open-ended, grounding-based, attribution- based, and summarizer. We consider 6000 data points in 8 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 total for this study and 1200 multimodal data instances for each category. To begin with, open-ended queries are normally much more complex and dynamic in clinical applications; therefore show a little downward slope (dipped) to 59% (F1) accuracy. Otherwise, all other categories remained relatively flat (between ~68% and 73.3%). This suggests that the suggested enhancement strategy scales reliably across diverse semantic multimodal cues rather than over-learned specific category. In the contrary, Figure 3(b), shows query complexity analysis over our C-GRPO based BioMed- Agent-RL. Here, we are considering 2000 clinical instances per complexities. It can be noted that a methodical decline in efficiency as clinical grounding and reasoning difficulty rises from easy to hard level cues. The accuracy of easy cues (73.08%) moves downward to hard cues (~58%), mirroring the drop across all benchmarks. Finally, the comparative efficiency study of query complexities and categories for our agent helps to establish a robust, reliable, explainable, and trustworthy clinical intelligence systems. More details about the performances on query level is depicted in Table 2. Figure 4 displays the qualitative result comparison among various checkpoints with C-GRPO approach. The goal is to analyze the quality of inference from 2 group of tasks, such as (a) VQA task, (b) report generation. VQA task refers to regular days to day clinical tasks though user interaction and regular dialogues by utilizing execution parameters <params> as βVQAβ to the backbone. On the other hand, report generation refers to generated inference through a specific format (βFindings:β and βImpressions :β). It also passes parameter (<params> as βreportβ) to the backbone. When asking a cues like, βany evidence of pneumonia?β The agent generates VQA task, DPO and CPO shows surface level output for clinical x-ray cue with acute diseases (e.g, βpneumoniaβ) such as, βpossible infectionβ, βearly infection,β, and βBilateral opacitiesβ. Whereas, our BioMed-Agent-RL replies with sematic clinical keywords like, βpatchy opacityβ, βright lower lobeβ, βpleural effusionβ, and βacute pneumoniaβ, grounding and reasoning with rectangular bounding boxes with format [x1, y1, x2, y2]. Based on captioning LLM, the modelβs prompt builder refers to the backbone with tag <API> which containing model name, for instance βMMedAgentβ, βDinov2β. Then the referred model will be invoked according to the referred tags. Collectively, these findings suggest that BioMed-Agent- RL efficiency enhancements are not only consistently distributed, but also specifically noticeable in clinical challenging semantic reasoning and grounding scenarios. V. FUTURE DIRECTION Even though, the BioMed-Agent-RL outperforms in critical clinical reasoning, grounding and segmentation task, it is not without flaws. First, It oftentimes prone to show low accuracy for long longitudinal multimodal cues. Therefore, a persistent memory augmentation with adaptive retrieval is necessary to deal with episodic cues. We have already planning to implement a persistent memory module for our agentic system. Secondly, when dealing with curriculum learning in GRPO, we have a plan to implement an uncertainty aware reward modelling and policy function to improve reliability of the agent. VI. CONCLUSION In this manuscript, we propose a curriculum learning on GRPO reinforcement learning based clinical agent for complex biomedical VQA and report generation tasks. The efficient reasoning and grounding approach is proposed across 5 semantic query categories, several biomedical benchmarks, and 3 complexity levels. Our C-GRPO approach reliably surpass DPO and CPO optimization approach up to maximum accuracy of 73.03% (with maximum gain of 10.5% compared to its closest RL approach (DPO)). The backbone-level studies confirms that the performance gains are architecture and algorithm agnostic. Furthermore, dataset-specific further validated that enhancements were steadily positive across diverse clinical tasks, indicating robust data generalization. Query semantic categories and complexity-level evaluation reveals that the agent generalize well for any uncommon and super complex clinical cues. The direction indicates that C-GRPO improves reasoning consistency under multi- stage and structural reasoning conditions, therefore reducing error distribution through reward modelling in clinical AI systems. REFERENCES [1] B. Njei et al., βArtificial intelligence agents in healthcare research: A scoping review,β PLoS One, vol. 21, no. 2, p. e0342182, Feb. 2026, doi: 10.1371/journal.pone.0342182. [2] W. Wang et al., βA Survey of LLM-based Agents in Medicine: How far are we from Baymax?,β May 26, 2025, arXiv: arXiv:2502.11211. doi: 10.48550/arXiv.2502.11211. [3] T. P. Quinn, M. Senadeera, S. Jacobs, S. Coghlan, and V. Le, βTrust and medical AI: the challenges we face and the expertise needed to overcome them,β Journal of the American Medical Informatics 9 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 Association, vol. 28, no. 4, p. 890β894, Mar. 2021, doi: 10.1093/jamia/ocaa268. [4] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, βProximal Policy Optimization Algorithms,β 2017, arXiv. doi: 10.48550/ARXIV.1707.06347. [5] R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, βDirect Preference Optimization: Your Language Model is Secretly a Reward Model,β Jul. 29, 2024, arXiv: arXiv:2305.18290. doi: 10.48550/arXiv.2305.18290. [6] G. E. Weissman, T. Mankowitz, and G. P. Kanter, βUnregulated large language models produce medical device-like output,β npj Digit. Med., vol. 8, no. 1, p. 148, Mar. 2025, doi: 10.1038/s41746-025- 01544-y. [7] B. Li et al., βMMedAgent: Learning to Use Medical Tools with Multi-modal Agent,β Oct. 05, 2024, arXiv: arXiv:2407.02483. doi: 10.48550/arXiv.2407.02483. [8] K. Zhu, P. Xia, Y. Li, H. Zhu, S. Wang, and H. Yao, βMMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization,β Jun. 04, 2025, arXiv: arXiv:2412.06141. doi: 10.48550/arXiv.2412.06141. [9] Y. Zhou, L. Song, and J. Shen, βImproving Medical Large Vision-Language Models with Abnormal- Aware Feedback,β Jun. 02, 2025, arXiv: arXiv:2501.01377. doi: 10.48550/arXiv.2501.01377. [10] P. Xia et al., βMMedAgent-RL: Optimizing Multi- Agent Collaboration for Multimodal Medical Reasoning,β Jan. 26, 2026, arXiv: arXiv:2506.00555. doi: 10.48550/arXiv.2506.00555. [11] DeepSeek-AI et al., βDeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning,β Nature, vol. 645, no. 8081, p. 633β638, Sep. 2025, doi: 10.1038/s41586-025-09422-z. [12] Z. Shao et al., βDeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models,β Apr. 27, 2024, arXiv: arXiv:2402.03300. doi: 10.48550/arXiv.2402.03300. [13] H. Deng, D. Zou, R. Ma, H. Luo, Y. Cao, and Y. Kang, βBoosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning,β Mar. 10, 2025, arXiv: arXiv:2503.07065. doi: 10.48550/arXiv.2503.07065. [14] A. Singh et al., βOpenAI GPT-5 System Card,β Dec. 19, 2025, arXiv: arXiv:2601.03267. doi: 10.48550/arXiv.2601.03267. [15] J. L. Popover et al., βArtificial Intelligence in Medicine: A Specialty-Level Overview of Emerging AI Trends,β JSLS, vol. 29, no. 3, p. e2025.00041, 2025, doi: 10.4293/JSLS.2025.00041. [16] A. Al-Zoghby, A. Ebada, A. Saleh, M. Abdelhay, and W. Awad, βA Comprehensive Review of Multimodal Deep Learning for Enhanced Medical Diagnostics,β CMC, vol. 84, no. 3, p. 4155β4193, 2025, doi: 10.32604/cmc.2025.065571. [17] D. Schouten et al., βNavigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications,β Medical Image Analysis, vol. 105, p. 103621, Oct. 2025, doi: 10.1016/j.media.2025.103621. [18] Z. Wang et al., βA Comprehensive Survey of LLM Alignment Techniques: RLHF, RLAIF, PPO, DPO and More,β Jul. 23, 2024, arXiv: arXiv:2407.16216. doi: 10.48550/arXiv.2407.16216. [19] T. C. Frommeyer et al., βReinforcement Learning and Its Clinical Applications Within Healthcare: A Systematic Review of Precision Medicine and Dynamic Treatment Regimes,β Healthcare (Basel), vol. 13, no. 14, p. 1752, Jul. 2025, doi: 10.3390/healthcare13141752. [20] J. Li et al., βAgent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents,β Jan. 17, 2025, arXiv: arXiv:2405.02957. doi: 10.48550/arXiv.2405.02957. [21] G. Cui et al., βThe Entropy Mechanism of Reinforcement Learning for Reasoning Language Models,β May 28, 2025, arXiv: arXiv:2505.22617. doi: 10.48550/arXiv.2505.22617. [22] C. Li et al., βLLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day,β Jun. 01, 2023, arXiv: arXiv:2306.00890. doi: 10.48550/arXiv.2306.00890. [23] H. Lu et al., βDeepSeek-VL: Towards Real-World Vision-Language Understanding,β Mar. 11, 2024, arXiv: arXiv:2403.05525. doi: 10.48550/arXiv.2403.05525. [24] J. J. Lau, S. Gayen, A. Ben Abacha, and D. Demner- Fushman, βA dataset of clinically generated visual questions and answers about radiology images,β Sci Data, vol. 5, no. 1, p. 180251, Nov. 2018, doi: 10.1038/sdata.2018.251. [25] B. Liu, L.-M. Zhan, L. Xu, L. Ma, Y. Yang, and X.- M. Wu, βSLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering,β Feb. 18, 2021, arXiv: arXiv:2102.09542. doi: 10.48550/arXiv.2102.09542. [26] X. He, Y. Zhang, L. Mou, E. Xing, and P. Xie, βPathVQA: 30000+ Questions for Medical Visual 10 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. x, NO. x, 2020 Question Answering,β Mar. 07, 2020, arXiv: arXiv:2003.10286. doi: 10.48550/arXiv.2003.10286. [27] A. E. W. Johnson et al., βMIMIC-CXR, a de- identified publicly available database of chest radiographs with free-text reports,β Sci Data, vol. 6, no. 1, p. 317, Dec. 2019, doi: 10.1038/s41597-019- 0322-0. [28] D. Demner-Fushman et al., βPreparing a collection of radiology examinations for distribution and retrieval,β Journal of the American Medical Informatics Association, vol. 23, no. 2, p. 304β310, Mar. 2016, doi: 10.1093/jamia/ocv080.