Paper deep dive

Learning to Present: Inverse Specification Rewards for Agentic Slide Generation

Karthik Ragunath Ananda Kumar, Subrahmanyam Arunachalam

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 55

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%

Last extracted: 3/22/2026, 5:47:24 AM

Summary

The paper introduces SlideRL, a reinforcement learning environment and multi-component reward architecture for automated slide generation. It utilizes an 'inverse specification reward' to ensure generated slides faithfully represent their intended purpose, and employs GRPO to fine-tune Qwen2.5-Coder-7B, achieving performance comparable to larger models like Claude Opus 4.6.

Entities (5)

Claude Opus 4.6 · model · 100%GRPO · algorithm · 100%Qwen2.5-Coder-7B · model · 100%SlideRL · dataset · 100%OpenEnv · framework · 95%

Relation Signals (3)

Qwen2.5-Coder-7B → trainedvia → GRPO

confidence 100% · Our approach fine-tunes a Qwen2.5-Coder-7B model via GRPO

SlideRL → containsdatafrom → Claude Opus 4.6

confidence 95% · We contribute SlideRL, an open-source dataset of 288 multi-turn rollout trajectories across all six evaluated models

SlideRL → supports → Slide Generation

confidence 90% · SlideRL, an open-source dataset of 288 multi-turn rollout trajectories

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Automated presentation generation remains a challenging task requiring coherent content creation, visual design, and audience-aware communication. This work proposes an OpenEnv-compatible reinforcement learning environment where LLM agents learn to research topics, plan content, and generate professional HTML slide presentations through tool use. We introduce a multi-component reward system combining structural validation, render quality assessment, LLM-based aesthetic scoring, content quality metrics, and an inverse specification reward that measures how faithfully generated slides convey their intended purpose. The inverse specification reward, an "inverse task" where an LLM attempts to recover the original specification from generated slides, provides a holistic quality signal. Our approach fine-tunes Qwen2.5-Coder-7B via GRPO, training only 0.5% of parameters on prompts derived from expert demonstrations collected using Claude Opus 4.6. Experiments on 48 diverse business briefs across six models demonstrate that our fine-tuned 7B model achieves 91.2% of Claude Opus 4.6's quality while improving 33.1% over the base model. The six-model comparison reveals that instruction adherence and tool-use compliance, rather than raw parameter count, determine agentic task performance. We contribute SlideRL, an open-source dataset of 288 multi-turn rollout trajectories across all six models: this https URL Code: this https URL

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

55,123 characters extracted from source content.

Expand or collapse full text

Learning to Present: Inverse Specification Rewards for Agentic Slide Generation Karthik Ragunath Ananda Kumar*, Tavus Inc., University of Texas at Dallas and Subrahmanyam Arunachalam*, Texas A&M University Abstract Automated presentation generation remains a challenging task requiring coherent content creation, visual design, and audience-aware communication. This work proposes an OpenEnv-compatible reinforcement learning environment where Large Language Model (LLM) agents learn to research topics, plan content, and generate professional HTML slide presentations through tool use. We introduce a multi-component reward system combining structural validation, render quality assessment, LLM-based aesthetic scoring, content quality metrics, and an inverse specification reward that measures how faithfully generated slides convey their intended purpose. The inverse specification reward, an “inverse task” where an LLM attempts to recover the original presentation specification from generated slides, provides a holistic quality signal. Our approach fine-tunes a Qwen2.5-Coder-7B model via GRPO, training only 0.5% of parameters on prompts derived from expert demonstrations collected using Claude Opus 4.6. Experiments on 48 diverse business presentation briefs across six models, including Claude Opus 4.6, Claude Sonnet 4.6, Llama 4 Scout, GPT OSS 120B, and base Qwen 7B, demonstrate that our fine-tuned 7B model achieves 91.2% of Claude Opus 4.6’s quality while improving 33.1% over the untuned base model. The six-model comparison reveals that instruction adherence and tool-use compliance, rather than raw parameter count, determine agentic task performance. The divide-and-conquer reward architecture provides interpretable quality assessment across six dimensions, supporting targeted improvements in agentic presentation generation. We contribute SlideRL, an open-source dataset of 288 multi-turn rollout trajectories across all six evaluated models, publicly available at https://huggingface.co/datasets/KarthikRagunathAnandaKumar/sliderl-multi-turn-rollouts. Code is available at https://github.com/pushing-the-frontier/slide-forge-llm. I Introduction The creation of professional presentations is a ubiquitous task in business, education, and research contexts. Despite advances in generative AI, automated slide generation remains challenging because it requires topic research, content structuring, visual design, and audience-aware communication, all coordinated through a multi-step workflow. Recent work in LLM agents has shown strong results in tool use and multi-step reasoning [24, 17]. However, training agents for complex creative tasks like presentation generation poses distinct challenges: (1) the action space is large—the agent must select from 14 tools and specify their parameters, (2) quality assessment requires multiple orthogonal criteria, (3) the task demands both factual accuracy and aesthetic appeal, and (4) slides must follow a coherent narrative arc with logical sequencing and temporal flow across the deck. We address these challenges with a reinforcement learning environment that frames presentation generation as a sequential decision-making problem. The environment exposes 14 tools organized into 5 categories—research (web_search, fetch_url), content planning (create_outline, revise_outline), design (generate_slide, edit_slide, set_theme), deck structure (get_slide_content, delete_slide, reorder_slides, duplicate_slide, insert_slide), and meta (review_deck, finalize)—through which the agent progresses across five phases: research, planning, generation, refinement, and finalization. As illustrated in Fig. 1, this decomposition divides the complex task into manageable phases while employing a reward architecture that evaluates quality across six dimensions. Figure 1: Architecture of the proposed system. The LLM agent working in the training loop generates tool calls that are executed in the environment, with multi-component rewards guiding policy optimization. Our key contributions are: 1. OpenEnv [10]-Compatible RL Environment: A reinforcement learning environment with 14 tools across 5 categories, supporting the full presentation creation workflow from research to finalization. 2. Multi-Component Reward System: A reward architecture combining six quality dimensions with configurable weights, allowing interpretable and targeted quality assessment. 3. Inverse Specification Reward (Novel): A new “inverse task” reward formulation in which an LLM attempts to reconstruct the original specification from the generated slides alone. To our knowledge, this is the first application of input-reconstruction as a reward signal for evaluating holistic coherence and faithfulness in the context of automated slide and presentation generation. 4. Dense Step Rewards: Quality-delta based step rewards that provide dense training signals rather than sparse episode-end rewards. 5. Multi-Format Output via Tool Use: The fine-tuned model learns to trigger appropriate tool calls that produce presentations in multiple output formats (HTML slide decks and PPTX files), enabling downstream consumption across web rendering and traditional presentation software without format-specific training. 6. Expert Trajectory Generation: A pipeline using Claude Opus 4.6 [2, 3] to generate high-quality trajectories for GRPO fine-tuning of smaller models. 7. SlideRL Dataset: We open-source a multi-turn rollout dataset of 288 complete trajectories (48 briefs × 6 models) with per-turn tool calls, environment observations, step rewards, and quality scores, publicly available at https://huggingface.co/datasets/KarthikRagunathAnandaKumar/sliderl-multi-turn-rollouts. I Related Work I-A LLM Agents and Tool Use Recent work on LLM agents has demonstrated effective tool use across a range of tasks [24, 17, 26]. ReAct [26] introduced the pattern of interleaving reasoning and acting, while Toolformer [17] showed that LLMs can learn to use tools through self-supervised learning. Our work extends these approaches to presentation generation, where tool use must be coordinated across research, content creation, and design phases. I-B Reinforcement Learning for LLMs RLHF [14] established the use of human feedback to align LLMs. Subsequent work has explored alternatives including DPO [16], GRPO [20], and various reward modeling approaches. Our work employs GRPO for its efficiency in fine-tuning with relative rewards, combined with a multi-component reward architecture tailored to this domain. I-C Automated Presentation Generation Prior work on automated presentation generation has focused on extractive slide generation [19], document-to-slide pipelines [5], and learning-based content selection [7]. Recent general-purpose LLM systems [12] show strong generative capability, but presentation-oriented methods still typically lack the structured reward signals needed for systematic improvement. Our work fills this gap with a multi-component reward architecture. I-D LLM-as-Judge for Quality Assessment Recent work has shown that LLMs can serve as reliable evaluators for generated content [27, 8]. Our approach extends this idea through the inverse specification reward, which uses an LLM to assess holistic quality by attempting to recover the original task specification from the generated output. I Environment Design I-A Overview The environment implements the OpenEnv [10] interface with standard reset() and step() methods. The environment maintains state across an episode, tracking research context, outline structure, generated slides, and workflow phase. Episode Lifecycle: 1. RESEARCH: Agent gathers information via web_search, fetch_url 2. PLAN: Agent structures content via create_outline, revise_outline 3. GENERATE: Agent creates slides via generate_slide 4. REFINE: Agent improves via edit_slide, review_deck 5. DONE: Agent calls finalize to end the episode I-B Action Space The action space consists of 14 tools organized into 5 categories, as shown in Table I. TABLE I: Tool categories and descriptions. Category Tools Description Research web_search, fetch_url Info gathering Content create_outline, revise_outline Planning Design generate_slide, edit_slide, set_theme Slide creation Structure get_slide_content, delete_slide, reorder_slides, duplicate_slide, insert_slide Deck ops Meta review_deck, finalize Quality control Each action is specified as a JSON object with tool name and parameters. The following examples, drawn from actual agent trajectories, illustrate tool calls across the four workflow phases: Research: ⬇ "tool": "web_search", "query": "electric vehicle market size 2026 global sales projections statistics" Planning: ⬇ "tool": "create_outline", "sections": [ "title": "Q4 2025 Financial Results", "bullet_points": [ "$142.3M revenue, 23% YoY growth", "ARR reaching $156.8M with 118% NRR", "Net income of $18.7M at 13.1% margin" ], "title": "Revenue & Profitability", "bullet_points": ["..."] ] Generation: ⬇ "tool": "generate_slide", "slide_idx": 0, "title": "Q4 2025 Financial Results", "sections": [ "heading": "Revenue Milestone", "body": "$142.3M in Q4 revenue -- up 23% YoY.", "heading": "ARR & Retention", "body": "ARR: $156.8M | NRR: 118% | Churn: 4.2%." ] Refinement: ⬇ "tool": "edit_slide", "slide_idx": 0, "title": "Q4 2025 Financial Results", "sections": [ "heading": "Revenue Milestone", "body": "$142.3M in Q4 revenue -- up 23% YoY.", "heading": "ARR & Retention", "body": "ARR: $156.8M | NRR: 118% | Churn: 4.2%. Deep customer stickiness." ] I-C State Representation The environment state captures the task-relevant information needed for decision-making and reward computation, as shown in Table I. TABLE I: Environment state representation. Field Type Description brief SlideBrief Task configuration research_context list[dict] Accumulated research outline list[dict] Slide structure slides_html list[str] Generated HTML slides_png list[bytes] Rendered PNGs theme str Visual theme phase str Workflow phase edit_mode bool Edit task flag original_slides_html list[str] Pre-edit HTML The implementation additionally tracks episode metadata (episode ID, step count, step budget, termination flag, accumulated reward) for environment bookkeeping; these are not part of the state representation exposed to the agent’s policy. I-D Observation Space After each action, the agent receives an observation containing the fields listed in Table I. TABLE I: Observation space fields. Field Type Description result str Tool execution result success bool Action succeeded current_slide_count int Slides created phase str Workflow phase slide_previews list[str] Base64 PNG thumbnails The LLM agent receives a text rendering of this observation at each step: Tool result (success=success): result State: phase=phase, slides=count/target, turns remaining=budget Tool results are concise confirmations. For example, generate_slide returns "Slide 3 generated and rendered (3 sections)." The agent relies on its conversation history to track progress across the episode. The environment returns the standard RL signals (step reward, termination flag, and step index) alongside the observation, following the Gymnasium (obs, reward, terminated, info) convention. IV Multi-Component Reward System The multi-component reward architecture evaluates presentation quality across six dimensions. Rather than attempting to capture quality in a single metric, we decompose it into interpretable components that can be independently assessed and optimized. IV-A Reward Components Table IV lists the six reward components with their weights. TABLE IV: Reward component weights and descriptions. Component Weight Description code_rules 1.0 Structural validation render_quality 2.0 Render success, HTML validity aesthetic_html 1.5 HTML design quality aesthetic_visual 1.5 Visual screenshot quality content_quality 2.0 Topic relevance, grounding spec_reconstruction 2.0 Inverse specification faithfulness The aggregate reward is computed as: Raggregate=∑iwi⋅ri∑iwiR_aggregate= _iw_i· r_i _iw_i (1) where wiw_i is the weight and ri∈[0,1]r_i∈[0,1] is the score for component i. IV-B Code Rules Reward The structural validation reward scores adherence to presentation conventions. For each slide, the score is computed as: rcode=1N∑j=1N(0.25⋅title,j+ssec,j+0.25⋅min⁡(wj,wt)max⁡(wj,wt)+0.25⋅nfilled,jntotal,j)r_code= 1N _j=1^N (0.25·1_title,j+s_sec,j\\ +0.25· (w_j,w_t) (w_j,w_t)+0.25· n_filled,jn_total,j ) (2) where ssec,js_sec,j scores section count adherence: 0.250.25 if the section count of slide j matches the target exactly, 0.100.10 partial credit if sections exist but the count differs, and 0 otherwise. The individual checks are: • Title present (0.25): .title element exists with text. • Section count (0.25/0.10): 0.25 if exact match to target sections per slide; 0.10 partial credit if sections exist but count differs. • Word count (0.25): 0.25×min⁡(w,wt)/max⁡(w,wt)0.25× (w,w_t)/ (w,w_t), ratio of actual to target. • Non-empty sections (0.25): 0.25×(nfilled/ntotal)0.25×(n_filled/n_total), fraction of sections containing text. IV-C Render Quality Reward This component assesses technical rendering success via three sub-components: rrender=0.4⋅min⁡(nslidesntarget,1)+0.3⋅nrenderednslides+0.3⋅vhtmlr_render=0.4· ( n_slidesn_target,1 )+0.3· n_renderedn_slides+0.3· v_html (3) where nslidesn_slides is the number of slides created, ntargetn_target is the target slide count from the brief, nrenderedn_rendered is the number of slides successfully rendered to PNG, and vhtml∈0,1v_html∈\0,1\ indicates whether required HTML elements are present. IV-D Aesthetic Rewards We employ LLM-based evaluation (Claude Opus 4.6) for aesthetic assessment. Each slide is scored independently from 0.0 to 1.0, then averaged across the deck. Results are cached by content hash to ensure deterministic scoring on repeated evaluations. HTML Structure Scoring (aesthetic_html): An LLM evaluates the raw HTML/CSS of each slide across four equally weighted dimensions (0.25 each): (1) layout and structure, including clear title/section hierarchy and logical organization; (2) content balance and appropriate density; (3) visual styling with modern CSS, color harmony, and typography; (4) professional polish with executive-ready, consistent formatting. Visual Scoring (aesthetic_visual): For rendered PNG screenshots (produced by Playwright), an LLM evaluates four equally weighted dimensions (0.25 each): (1) visual design with color harmony, contrast, and modern aesthetics; (2) layout and spacing, including whitespace, alignment, and organization; (3) typography with font hierarchy, readability, and density; (4) professional polish with executive-ready appearance and consistency. These LLM-as-judge approaches capture design principles that are difficult to encode in rule-based metrics. IV-E Content Quality Reward Content quality is assessed across four dimensions: topic relevance (weight 0.35, slides mentioning topic words), factual grounding (0.25, overlap with research results), content uniqueness (0.20, ratio of unique slides), and narrative flow (0.20, outline coverage). IV-F Inverse Specification Reward The inverse specification reward measures how faithfully the generated slides convey their intended purpose. The idea is simple: given only the output, can we recover the input specification? Given a completed slide deck, we prompt an LLM to predict the original brief: ⬇ Given the slide deck, predict: "topic": "...", "audience": "...", "num_slides": N, "key_themes": ["...", "..."] The reconstruction score compares predictions against the actual brief: rrecon=0.40⋅stopic+0.25⋅saudience+0.15⋅scount+0.20⋅sthemesr_recon=0.40· s_topic+0.25· s_audience+0.15· s_count+0.20· s_themes (4) where each sub-score measures overlap between predicted and actual values: • Topic similarity (0.40): Word overlap between predicted and actual topic. • Audience match (0.25): Exact match, partial match, or word overlap. • Slide count accuracy (0.15): Ratio of predicted to actual count. • Theme coverage (0.20): Overlap between predicted themes and topic words. A presentation that clearly communicates its purpose will allow accurate specification reconstruction; a confused or off-topic presentation will not. V Training Pipeline V-A Expert Trajectory Generation We generate expert trajectories using Claude Opus 4.6 [2, 3] as the agent. Each trajectory is a complete episode from research through finalization. Figure 2: Expert trajectory generation pipeline. The expert LLM generates a tool call each turn, which is executed in the environment. Step rewards are computed as quality deltas after each action, and the conversation history accumulates until the episode terminates. The system prompt guides the expert through the workflow phases, requiring exactly one JSON tool call per turn. V-B Dense Step Rewards Rather than sparse episode-end rewards, we compute dense step rewards as quality deltas: rstep=(Qnew−Qold)+ractionr_step=(Q_new-Q_old)+r_action (5) where Q is the aggregate quality score and ractionr_action provides small bonuses/penalties for action success/failure (+0.01+0.01 for successful actions, +0.1+0.1 for successful finalization, −0.02-0.02 for failed actions). This formulation corresponds to potential-based reward shaping [11], where the shaping function F(s,s′)=γΦ(s′)−Φ(s)F(s,s )=γ (s )- (s) uses Φ(s)=Qaggregate(s) (s)=Q_aggregate(s) as the potential function. This class of shaping is guaranteed to preserve the optimal policy while providing dense signal. Motivation for Dense Rewards. Presentation generation episodes span 20–35 turns, with the final quality only observable after finalize is called. Sparse episode-end rewards create a severe credit assignment problem: which of the 30+ actions contributed to success? Dense step rewards address this through: (1) immediate feedback—each action receives a reward signal based on quality improvement, enabling faster learning convergence; (2) credit assignment—the quality delta directly attributes reward to the action that caused the change; (3) noise reduction—multiple smaller reward signals partially cancel noise across steps; (4) exploration guidance—negative deltas discourage actions that degrade quality, while positive deltas reinforce productive actions. V-C Reward Function Properties and Theoretical Justification Our reward system is both stochastic and non-differentiable. Environment execution involves discrete operations (HTML parsing, conditional logic), LLM-as-judge scoring requires black-box API calls, and rule-based checks involve binary conditions. LLM scoring also exhibits slight variations across calls. This motivates our choice of GRPO over supervised methods. The theoretical justification rests on the policy gradient theorem [22]: ∇θJ(θ)=τ∼πθ[R(τ)⋅∇θlog⁡πθ(τ)] _θJ(θ)=E_τ _θ [R(τ)· _θ _θ(τ) ] (6) where J(θ)J(θ) is the expected reward, τ is a trajectory (token sequence), R(τ)R(τ) is the scalar reward, and πθ(τ) _θ(τ) is the policy probability. Critically, the gradient operator ∇θ _θ acts only on log⁡πθ(τ) _θ(τ), not on R(τ)R(τ). The reward passes through the gradient operator untouched; it is a scalar weight on the policy gradient, never differentiated through. Variance analysis. While non-differentiable rewards preserve gradient correctness in expectation, they introduce variance. For a group of K completions with rewards R1,…,RKR_1,…,R_K, each decomposable as Ri=Ri∗+ηiR_i=R_i^*+ _i where ηi∼(0,ση2) _i (0, _η^2) represents evaluation noise, the signal-to-noise ratio of the advantage estimates is: SNR=σR∗2ση2SNR= _R^*^2 _η^2 (7) where σR∗2 _R^*^2 is the variance of true reward spread. When SNR<1SNR<1, noise dominates and learning becomes unreliable. Our multi-component reward system mitigates this through noise diversification: given C independent reward components with individual noise σc _c, the aggregate noise is: σagg=1W∑cwc2σc2 _agg= 1W _cw_c^2 _c^2 (8) where W=∑cwcW= _cw_c. Three of our six components (code rules, render quality, content quality) are nearly deterministic (σ≈0σ≈ 0), which substantially reduces aggregate noise relative to the stochastic LLM-based components (σ≈0.10σ≈ 0.10). With our weights, the aggregate noise (σagg≈0.03 _agg≈ 0.03) is an order of magnitude smaller than any individual LLM-based component. Caching LLM-as-judge scores by content hash eliminates stochasticity on repeated evaluations, making rewards deterministic for identical inputs. V-D GRPO Loss Function We employ Group Relative Policy Optimization (GRPO) [20], implemented via the TRL library [23], which extends the PPO clipped surrogate objective [18] with group-relative advantage normalization. The loss computation proceeds in three stages. Stage 1: Advantage computation. For each prompt, the model generates K completions. Each completion τk _k is executed in the environment and scored by the aggregate reward function, yielding scalar rewards R1,…,RKR_1,…,R_K. Advantages are computed via group normalization: Ak=Rk−μGσG+ϵadv A_k= R_k- _G _G+ _adv (9) μG=1K∑k=1KRk,σG=1K∑k=1K(Rk−μG)2 _G= 1K _k=1^KR_k, _G= 1K _k=1^K(R_k- _G)^2 Here, ϵadv _adv is a small numerical-stability constant. This group-mean baseline provides significant variance reduction: by centering rewards within each group, the advantage converts “everything is good” signals into contrastive “this completion was better than that one” signals. In our configuration, K=2K=2, yielding binary advantages of ±1± 1 after normalization. Stage 2: Per-token ratio computation. For each token ata_t in completion τk _k, we compute the importance sampling ratio: ρt=exp⁡(log⁡πθ(at∣a1:t−1,x)−log⁡πθold(at∣a1:t−1,x)) _t= \! ( _θ(a_t a_1:t-1,x)- _ _old(a_t a_1:t-1,x) ) (10) where x is the prompt, πθ _θ is the current model, and πθold _ _old is the frozen snapshot from when completions were generated. The per-token log-probability is: log⁡πθ(at∣a1:t−1,x)=zat−log∑v∈ezv _θ(a_t a_1:t-1,x)=z_a_t- _v e^z_v (11) where zvz_v are the logits and V is the vocabulary. Stage 3: Clipped surrogate loss. The per-token loss applies the PPO clip: ℒt=−min⁡(ρt⋅Ak,clip(ρt, 1−ϵclip, 1+ϵclip)⋅Ak)L_t=- \! ( _t· A_k,\;clip( _t,\,1- _clip,\,1+ _clip)· A_k ) (12) with ϵclip=0.2 _clip=0.2. The full GRPO loss includes an optional KL divergence penalty against a reference policy: ℒ=1|ℬ|∑k∈ℬ∑tℒt⋅mt∑tmt+β⋅DKL(πθ∥πref)L= 1|B| _k _tL_t· m_t _tm_t+β· D_KL ( _θ\,\|\, _ref ) (13) where mtm_t is a mask excluding padding tokens and β controls the strength of the KL penalty. In our configuration, β=0.0β=0.0, so no reference model is loaded and the KL term vanishes. The clipping mechanism alone constrains per-step policy updates. As discussed in Section VII-D, this proved sufficient for a short training horizon (200 steps on curated data) but insufficient for extended training (1000 steps on the full dataset), where cumulative policy drift led to mode collapse. GRPO reward function. The reward function bridges the RL objective with the environment by extracting tool calls from model completions, executing them, and computing aggregate scores: ⬇ function presentation_reward(completions, briefs): for each completion: 1. Reset environment with brief 2. Parse completion -> extract JSON 3. Score based on outcome: - No valid JSON -> -2.0 - Valid JSON, fail -> -1.0 - Valid JSON, success: -> compute aggregate_rewards(state) return scores The graduated penalty structure (−2.0-2.0 for unparseable output, −1.0-1.0 for failed execution, positive for successful actions) creates a curriculum effect: the model first learns to produce valid JSON tool calls, then learns to produce calls that succeed, then optimizes for quality. V-E Model Architecture and Parameter-Efficient Fine-Tuning Table V summarizes the GRPO training configuration. TABLE V: GRPO training configuration. Parameter Value Base model Qwen2.5-Coder-7B-Instruct Parameters (total) 7.61B Quantization 4-bit NormalFloat LoRA rank (r) 16 LoRA alpha (α) 16 LoRA dropout 0.0 Target modules q, k, v, o, gate, up, down proj Trainable parameters ∼ 40M (∼ 0.5% of total) Learning rate 5×10−55× 10^-5 Max sequence length 8,192 Max completion length 1,024 Num generations (K) 2 Clip epsilon (ϵclip _clip) 0.2 KL coefficient (β) 0.0 Training steps (selected model) 200 We apply Low-Rank Adaptation (LoRA) [6] to the base Qwen2.5-Coder-7B-Instruct model [15], which consists of 28 transformer blocks with Grouped Query Attention [1] (28 query heads, 4 key-value heads) and SwiGLU [21] feed-forward networks. LoRA adapters are attached to seven linear projections per block, covering both the attention mechanism and the feed-forward network: Attention projections (WQ,WK,WV,WOW_Q,W_K,W_V,W_O): These control what contextual patterns the model attends to, what information is extracted, and how multi-head outputs are combined. Adapting these projections lets the model learn task-specific attention patterns—for example, focusing on the brief’s topic keywords when generating slide content, or attending to previous tool results when planning the next action. Feed-forward projections (Wgate,Wup,WdownW_gate,W_up,W_down): The SwiGLU network controls feature detection and transformation. Adapting these projections lets the model develop task-specific representations, such as distinguishing between presentation phases or recognizing when to transition from research to content generation. For each adapted layer, LoRA decomposes the weight update as: W′=W+αrBAW =W+ αrBA (14) where W∈ℝdout×dinW ^d_out× d_in is the frozen pre-trained weight (stored in 4-bit), A∈ℝr×dinA ^r× d_in and B∈ℝdout×rB ^d_out× r are the trainable low-rank matrices, and r=16r=16 is the bottleneck rank. With α=r=16α=r=16, the scaling factor is unity. This yields approximately 40 million trainable parameters (0.5% of total), while the remaining 7.57 billion parameters remain frozen in 4-bit quantized format. The 4-bit quantization reduces the base model’s memory footprint from approximately 15 GB (float16) to 4 GB, enabling training on a single GPU. Figure 3: Architecture of the base Qwen2.5-Coder-7B-Instruct model. All 7.62B parameters are frozen and stored in 4-bit quantized format. The model comprises 28 transformer decoder layers, each containing Grouped-Query Attention (28 query heads, 4 KV heads, head dim 128) and a SwiGLU feed-forward network (intermediate dim 18,944). Legend: ∗ frozen layers, ∗ trainable layers. Figure 4: Architecture of the GRPO-finetuned SlideRL model. LoRA adapters (rank r=16r=16) are injected into all 7 linear projections per layer—Q, K, V, O (attention) and gate, up, down (FFN)—adding 1.44M trainable parameters per layer (40.4M total, 0.53% of 7.62B). Base weights remain frozen in 4-bit; only the LoRA matrices (bfloat16) are updated during GRPO training. Legend: ∗ frozen layers, ∗ trainable layers. Frozen components. Token embeddings (151,936×3,584151,936× 3,584), RMSNorm layers, rotary position embeddings (RoPE), and the language model head remain at their pre-trained values. These components encode general language capabilities that transfer directly to the presentation generation task without modification. VI Experiments VI-A Dataset We evaluate on 48 diverse business presentation briefs spanning: financial reports (Q4 results, budget allocation), investor pitches (Series A/B funding), market analyses (EV, cloud computing, fintech), technical reviews (cybersecurity, MLOps, DevOps), and strategic planning (M&A, product roadmaps). Briefs vary in target slides (6–10), audience (board, VCs, executives, engineers), confidence (0.3–1.0), and content type (structured data vs. open-ended topics). VI-B Evaluation Protocol We evaluate six models on identical briefs using the same environment and reward pipeline, as listed in Table VI, including Claude Opus 4.6 [2, 3] and Claude Sonnet 4.6 [4]. TABLE VI: Models evaluated. Model Type Params Fine-tuned (Ours) LoRA Qwen2.5-7B 7B (0.5% train.) Base Qwen 7B Qwen2.5-7B-Inst. 7B Claude Opus 4.6 Proprietary Undisclosed Claude Sonnet 4.6 Proprietary Undisclosed Llama 4 Scout Open-weight 109B (17B active) GPT OSS 120B Open-weight 120B For each model, the protocol is: (1) load brief from evaluation set, (2) run episode (max 35 turns) with the model’s agent loop, (3) compute quality scores using the multi-component reward system, (4) export deck.html and deck.pptx for manual review. The fine-tuned and base models run locally on an H100 GPU. All other models are served through hosted inference APIs. VI-C Results Table VII presents the aggregate results across all 48 briefs. TABLE VII: Aggregate results on 48 business briefs. Metric Fine-tuned (Ours) Base Qwen Claude Opus 4.6 Claude Sonnet 4.6 Llama 4 Scout GPT OSS 120B Overall quality 0.724 0.544 0.794 0.775 0.779 0.249 Completion rate 46/48 (95.8%) 34/48 (70.8%) 48/48 (100%) 48/48 (100%) 48/48 (100%) 15/48 (31.2%) Avg turns used 22.3 19.1 27.8 29.2 18.3 6.5 Avg slides created 7.0 5.2 7.3 7.3 7.3 2.2 Avg time/brief 71.6s 43.8s 393.3s 421.7s 155.4s 66.1s Fig. 5 ranks all six models by overall quality. The fine-tuned 7B model (0.724) achieves 91.2% of Claude Opus 4.6’s quality (0.794) while matching the smallest parameter tier in the comparison. Llama 4 Scout (0.779) emerges as a surprisingly strong baseline, approaching Claude Opus despite being a smaller open-weight model. GPT OSS 120B (0.249) performed poorly due to systematic failure to follow the required tool-call format, resulting in only 31.2% completion rate. Fig. 6 visualizes the quality–cost tradeoff. Figure 5: Model ranking by overall quality. Figure 6: Quality vs. inference cost. Table VIII shows per-component quality scores. TABLE VIII: Per-component quality scores. Component Fine-tuned Base Qwen Claude Opus Claude Sonnet Llama 4 Scout GPT OSS 120B code_rules 0.905 0.663 0.960 0.931 0.949 0.294 render_quality 0.958 0.708 1.000 1.000 1.000 0.309 content_quality 0.783 0.604 0.878 0.884 0.903 0.270 aesthetic_html 0.658 0.492 0.761 0.727 0.660 0.241 aesthetic_visual 0.539 0.397 0.568 0.550 0.546 0.184 spec_reconstruction 0.530 0.412 0.616 0.567 0.615 0.199 Figs. 7 and 8 present radar and grouped bar comparisons of component scores. Figure 7: Reward component comparison (radar chart). Figure 8: Quality scores by component. VI-D Analysis Impact of GRPO fine-tuning. Comparing the fine-tuned model against the base Qwen model isolates the effect of reinforcement learning. GRPO training produced a +33.1+33.1% improvement in overall quality (0.544→0.7240.544→ 0.724), a +25+25 percentage-point increase in completion rate (70.8%→95.8%70.8\%→ 95.8\%), and improved every reward component, most dramatically code_rules (+36.5+36.5%) and render_quality (+35.3+35.3%). Fig. 9 summarizes these operational improvements. Figure 9: Operational metrics comparison. Model tier analysis. The six-model comparison reveals a clear tier structure: • Tier 1 (q>0.77q>0.77): Claude Opus 4.6 (0.794), Llama 4 Scout (0.779), Claude Sonnet 4.6 (0.775)—all achieve 100% completion. • Tier 2 (q≈0.72q≈ 0.72): Fine-tuned Qwen 7B (0.724)—95.8% completion with competitive structural metrics. • Tier 3 (q≈0.54q≈ 0.54): Base Qwen 7B (0.544)—70.8% completion, demonstrating the value of GRPO. • Tier 4 (q<0.25q<0.25): GPT OSS 120B [13] (0.249)—despite 120B parameters, failed to follow the required JSON format, highlighting that parameter count alone does not determine agentic task performance. Parameter efficiency. Our fine-tuned 7B model achieves 91.2% of Claude Opus quality and 93.0% of Llama 4 Scout’s quality (0.724 vs. 0.779), despite having 15×15× fewer active parameters than Llama 4 Scout and training only 0.5% of its weights. On structural metrics, the fine-tuned model nearly matches Llama 4 Scout (code_rules 0.905 vs. 0.930, render_quality 0.958 vs. 1.000), demonstrating that GRPO fine-tuning closes most of the gap on tool-calling discipline. Llama 4 Scout’s remaining advantage is concentrated in content_quality (0.903 vs. 0.783), attributable to its larger active parameter budget for content synthesis. This positions Llama 4 Scout [9] as a promising candidate for future GRPO fine-tuning. Gap to the expert model. The fine-tuned model achieves 91.2% of Claude Opus’s overall quality (0.724 vs. 0.794). The gap is concentrated in content_quality (0.783 vs. 0.878) and spec_reconstruction (0.530 vs. 0.616), suggesting limited capacity for deep content synthesis at 7B parameters. Structural metrics (code_rules 0.905 vs. 0.960, render_quality 0.958 vs. 1.000) are near-parity. Head-to-head competitiveness. Against the base Qwen 7B model, the fine-tuned model wins decisively (34W/2T/12L). Against Tier 1 models, losses are predominantly small-margin, indicating that the quality gap narrows on easier briefs, while wins demonstrate that a 7B model can outperform much larger models on specific brief types. Outright wins over all models. On 5 of 48 briefs, the fine-tuned 7B model ranks #1 outright, as shown in Table IX. TABLE IX: Briefs where the fine-tuned model outperforms all competitors. Brief Ours Next Best Δ Cloud Cost Optimization 0.836 Sonnet 4.6 (0.788) +0.048+0.048 Content Marketing ROI 0.826 Opus 4.6 (0.824) +0.002+0.002 Customer Success Metrics 0.816 Opus 4.6 (0.807) +0.009+0.009 B2B Sales Automation 0.800 Opus 4.6 (0.770) +0.030+0.030 Edge Computing Analysis 0.792 Base Qwen (0.781) +0.011+0.011 On 4 of these 5 winning briefs the fine-tuned model beats Claude Opus 4.6, the same model family that serves as LLM-as-judge for the aesthetic and content quality reward components. This rules out judge-bias as an explanation: if anything, using Claude Opus as both the expert trajectory generator and the evaluator should favor Claude Opus. Across all 48 evaluation briefs, the fine-tuned 7B model beats Claude Opus 4.6 [2, 3]—currently the state-of-the-art in code generation—on 12 briefs (25%), despite having orders of magnitude fewer parameters. Areas for improvement: (1) Content depth—the content_quality gap (0.783 vs. Llama 4 Scout’s 0.903 and Claude Opus’s 0.878) is the largest deficit; (2) brief faithfulness—reconstruction scores (0.530 vs. 0.616) indicate occasional topic drift; (3) aesthetic quality—HTML aesthetic scores lag behind Tier 1 models (0.658 vs. Claude Opus 0.761). VI-E Effect of Training Steps and Dataset Scale Table X summarizes the effect of training steps and dataset scale. TABLE X: Effect of training steps and dataset scale. Run Dataset Steps Aggregate Compl. Rate Curated 3 traj. 100 0.623 71.2% Curated 3 traj. 200 0.689 82.4% Scaled 48 traj. 200 0.724 95.8% Scaled 48 traj. 300 0.0 0% Scaled 48 traj. 1000 0.0 0% The curated run (3 high-quality expert trajectories, 200 steps) produced a viable model. The scaled run (48 trajectories, 1000 steps) achieved its best performance at checkpoint-200 (0.724, 95.8% completion) before exhibiting complete mode collapse at checkpoints beyond step 200 (see Section VII-D). Notably, the scaled run at 200 steps outperformed the curated run (0.724 vs. 0.689), indicating that increased dataset diversity improves early-stage learning. However, the same run collapsed into reward hacking at longer horizons. VII Discussion VII-A Divide and Conquer Reward Architecture The multi-component reward system has several practical advantages: (1) interpretability—each component measures a distinct quality dimension; (2) flexibility—weights can be adjusted to prioritize different aspects; (3) robustness—failure in one component does not prevent training; (4) noise diversification—as analyzed in Section V, the combination of deterministic and stochastic reward components reduces aggregate evaluation noise. VII-B Inverse Specification as Quality Signal The inverse specification reward captures coherence at the presentation level. Unlike component-wise metrics, it measures whether the presentation as a whole communicates its intended message. This inverse-task approach has several concrete benefits: (1) end-to-end assessment that captures properties component-wise metrics miss; (2) audience awareness, implicitly rewarding appropriate tone and complexity; (3) topic coherence, penalizing presentations that drift from the intended subject; (4) generalization to other tasks where output should faithfully reflect input specifications. VII-C On Non-Differentiable Rewards and Training Dynamics A distinctive aspect of our approach is that the GRPO training loss curve is not expected to decrease monotonically, even under successful convergence. This arises from three properties: (1) the PPO-style clip constrains the loss to a narrow band; (2) group-relative advantages remain zero-mean regardless of absolute quality; (3) online generation introduces batch-to-batch variation. Consequently, the appropriate convergence indicators are the reward curves (which should trend upward and stabilize) and completion rates (which should increase), rather than the loss itself. This is consistent with the general behavior of policy gradient methods [18]. Practical variance considerations. With K=2K=2 generations per prompt, the group normalization produces binary advantages (±1± 1), losing all magnitude information. Increasing K to 4–8 would yield richer advantage distributions at the cost of proportionally more compute. The standard error of the group mean scales as σR/K _R/ K, so quadrupling K halves the advantage noise. Role of the clip without KL regularization. Our configuration uses β=0.0β=0.0. The only constraint preventing arbitrary policy drift is the per-step clip (ϵclip=0.2 _clip=0.2). While each individual step is bounded, the cumulative effect over many steps can move the policy substantially from the pre-trained initialization. As detailed in Section VII-D, scaling to 1000 steps resulted in catastrophic mode collapse, demonstrating that the clip mechanism alone is insufficient for extended training and that introducing a KL coefficient (β>0β>0) is necessary for longer training horizons. Figure 10: GRPO training loss curve for the scaled 48-trajectory run. The x-axis represents training steps; the y-axis represents the GRPO loss (clipped surrogate policy gradient loss). Consistent with the analysis above, the loss does not decrease monotonically—it oscillates within a narrow band due to the clip constraint, group-relative advantage re-centering, and online completion generation. VII-D Observed Reward Hacking and Mode Collapse We conducted two separate GRPO training runs: (1) a curated run on 3 high-quality expert trajectories (200 steps), and (2) a scaled run on all 48 expert trajectories (1000 steps). While the scaled run produced a viable checkpoint at step 200 (selected for evaluation), it exhibited a pervasive failure mode at later checkpoints. At checkpoint-1000, the model called review_deck on every turn (35/35), producing zero slides and 0.0 aggregate quality, while accumulating a small positive cumulative reward of 0.35. At checkpoint-300, the model produced two initial productive actions before falling into the same loop for the remaining 33 turns. This represents a compound failure: reward hacking (exploiting the review_deck tool’s unconditional success signal) driving mode collapse (the action distribution collapsing to a single tool). The mechanism: review_deck always returns success=True regardless of deck state, earning +0.01+0.01 per step. More productive tools carry failure risk and negative rewards. Table XI shows the training reward trajectory. TABLE XI: Training reward trajectory (scaled 48-trajectory run). Steps Avg Min Max Diagnosis 0–49 −0.953-0.953 −1.188-1.188 −0.750-0.750 Exploration 50–99 −0.953-0.953 −1.125-1.125 −0.750-0.750 Still exploring 100–149 −0.933-0.933 −1.188-1.188 −0.625-0.625 Early learning 150–199 −0.655-0.655 −1.063-1.063 −0.250-0.250 Rapid improvement 200–249 −0.593-0.593 −0.875-0.875 −0.188-0.188 Residual diversity 250–299 −0.375-0.375 −0.750-0.750 −0.063-0.063 Degenerate emerging 300–349 −0.238-0.238 −0.688-0.688 0.0000.000 Collapse underway 350–399 −0.125-0.125 −0.375-0.375 0.0000.000 Variance narrowing 400–449 −0.105-0.105 −0.313-0.313 0.0000.000 Collapse entrenched 500+ −0.08-0.08 to −0.13-0.13 −0.375-0.375 0.0000.000 Full collapse Figure 11: Training reward curve for the scaled 48-trajectory GRPO run. The x-axis represents training steps; the y-axis represents the mean environment reward per step. The model exhibits consistent reward improvement from ≈−1.0≈-1.0 toward 0.00.0, demonstrating that GRPO drives meaningful policy refinement even in complex agentic settings. Early-to-mid training checkpoints (steps 100–200) capture the most behaviorally diverse and useful policies before variance narrows in later stages. The reward improved steadily from −1.0-1.0 toward 0.00.0 (Fig. 11), confirming that GRPO produces a clear learning signal in this agentic setting. Intermediate checkpoints from the high-variance region (steps 100–300) proved particularly valuable, capturing policies that balance exploration with tool-use competence and serving as strong starting points for downstream evaluation. A misleading diagnostic. We initially hypothesized that high reward variance at step 300 indicated healthy behavioral diversity. Empirical evaluation disproved this: the apparent variance was driven by residual base model behavior, not by learned diversity. The 0.0000.000 max rewards reflected successful review_deck calls (no state change), not successful slide creation. Root cause analysis. Three factors contributed to the collapse: (1) insufficient KL regularization (β=0.0β=0.0); (2) reward misspecification—the +0.01+0.01 per-step success bonus created a local optimum for no-risk tools; (3) binary advantage limitation with K=2K=2. This observation has direct implications for reward function design in agentic RL: tools that provide status information without modifying state should either carry an explicit cost or have diminishing returns to prevent reward hacking via no-op loops. VII-E Parameter Efficiency of LoRA Adaptation The LoRA configuration adapts only 0.5% of the model’s parameters while achieving competitive quality scores. This efficiency arises from two factors: (1) the behavioral shift from general-purpose code generation to presentation-specific tool calling is well-captured by rank-16 corrections; (2) the base model already possesses strong JSON generation, HTML understanding, and instruction-following capabilities that transfer directly. The frozen 4-bit base weights contribute to memory efficiency: the full training setup fits within a single GPU. VII-F Limitations 1. Computational cost of reward evaluation: Multiple LLM API calls per training step increase wall-clock time and cost. Reward model distillation could substantially reduce this overhead. 2. Reward hacking risk: As demonstrated in Section VII-D, tools with unconditional success signals can be exploited by the policy. 3. Domain specificity: Current reward functions are calibrated for business presentations; adaptation to other domains requires recalibration. 4. Group size limitation: With K=2K=2, advantage estimates are binary, limiting training signal quality. VII-G Future Work Key directions include: (1) scaling K to 4–8 for richer advantage distributions; (2) reward model distillation for deterministic, fast reward signals; (3) KL-regularized training (β>0β>0) for drift protection; (4) mode collapse mitigation via repetition penalties, diminishing returns for read-only tools, and terminal reward dominance; (5) early stopping on reward plateau; (6) human feedback integration; (7) multi-modal generation including image synthesis; (8) curriculum learning from simple to complex briefs; (9) upgrading to Qwen3 [25] as the base model; (10) cross-domain transfer of the inverse specification reward paradigm. VIII Conclusion We presented a reinforcement learning approach for training LLM agents to generate professional presentations. Our multi-component reward architecture enables interpretable quality assessment across six orthogonal dimensions with configurable weights. The inverse specification reward, an inverse task where an LLM recovers the original specification from generated output, provides a unique holistic quality signal that captures coherence properties missed by component-wise metrics. On the optimization side, we demonstrated that GRPO with non-differentiable, stochastic rewards is theoretically sound and practically effective. The policy gradient theorem guarantees that reward non-differentiability does not compromise gradient correctness; the multi-component architecture provides noise diversification; and LoRA adaptation achieves competitive quality while training only 0.5% of parameters. Experiments on 48 diverse business briefs across six models show that our fine-tuned Qwen2.5-7B model achieves 91.2% of Claude Opus 4.6’s quality score (0.724 vs. 0.794) while improving 33.1% over the base model (0.544). The broader comparison reveals that Llama 4 Scout (0.779) approaches Claude Opus quality at 2.5×2.5× faster inference (155s vs. 393s per brief), while GPT OSS 120B (0.249) demonstrates that raw parameter count does not guarantee agentic competence without instruction adherence. The divide-and-conquer approach to reward design offers a general framework applicable to other creative generation tasks. We release the environment, reward functions, and training pipeline at https://github.com/pushing-the-frontier/slide-forge-llm. We additionally open-source SlideRL (https://huggingface.co/datasets/KarthikRagunathAnandaKumar/sliderl-multi-turn-rollouts), containing 288 full-episode trajectories (48 briefs × 6 models) with per-turn tool calls, environment observations, step rewards, and final quality scores. References [1] J. Ainslie, J. Lee-Thorp, M. de Jong, Y. Zemlyanskiy, F. Lebrón, and S. Sanghai (2023) GQA: training generalized multi-query transformer models from multi-head checkpoints. arXiv preprint arXiv:2305.13245. Cited by: §V-E. [2] Anthropic (2025-05) Introducing Claude 4. Note: Anthropic Bloghttps://w.anthropic.com/news/claude-4 Cited by: item 6, §V-A, §VI-B, §VI-D. [3] Anthropic (2026-02) Claude Opus 4.6 system card. Note: Anthropichttps://w.anthropic.com/claude-opus-4-6-system-card Cited by: item 6, §V-A, §VI-B, §VI-D. [4] Anthropic (2026-02) Claude Sonnet 4.6 system card. Note: Anthropichttps://w.anthropic.com/claude-sonnet-4-6-system-card Cited by: §VI-B. [5] T. Fu, W. Y. Wang, D. McDuff, and Y. Song (2022) DOC2PPT: automatic presentation slides generation from scientific documents. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: §I-C. [6] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, Cited by: §V-E. [7] Y. Hu and X. Wan (2015) PPSGen: learning-based presentation slides generation for academic papers. IEEE Transactions on Knowledge and Data Engineering 27 (4), p. 1085–1097. Cited by: §I-C. [8] Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu (2023) G-Eval: NLG evaluation using GPT-4 with better human alignment. In Proceedings of EMNLP, Cited by: §I-D. [9] Meta AI (2025-04) The Llama 4 herd: the beginning of a new era of natively multimodal AI innovation. Note: Meta AI Bloghttps://ai.meta.com/blog/llama-4-multimodal-intelligence/ Cited by: §VI-D. [10] Meta PyTorch (2025) OpenEnv: agentic execution environments. Note: GitHubhttps://github.com/meta-pytorch/OpenEnv Cited by: item 1, §I-A. [11] A. Y. Ng, D. Harada, and S. Russell (1999) Policy invariance under reward transformations: theory and application to reward shaping. In International Conference on Machine Learning, Cited by: §V-B. [12] OpenAI (2023) GPT-4 technical report. arXiv preprint arXiv:2303.08774. Cited by: §I-C. [13] OpenAI (2025) gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925. Cited by: 4th item. [14] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. (2022) Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, Cited by: §I-B. [15] Qwen Team (2024) Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186. Cited by: §V-E. [16] R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn (2023) Direct preference optimization: your language model is secretly a reward model. In Advances in Neural Information Processing Systems, Cited by: §I-B. [17] T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom (2023) Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, Cited by: §I, §I-A. [18] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: §V-D, §VII-C. [19] A. Sefid, J. Wu, P. Mitra, and C. L. Giles (2019) Automatic slide generation for scientific papers. In Proceedings of the Third International Workshop on Capturing Scientific Knowledge (SciKnow), co-located with K-CAP, Cited by: §I-C. [20] Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y.K. Li, Y. Wu, and D. Guo (2024) DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §I-B, §V-D. [21] N. Shazeer (2020) GLU variants improve transformer. arXiv preprint arXiv:2002.05202. Cited by: §V-E. [22] R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour (1999) Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems, Cited by: §V-C. [23] L. von Werra et al. (2020) TRL: transformer reinforcement learning. Note: Hugging Facehttps://github.com/huggingface/trl Cited by: §V-D. [24] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. V. Le, and D. Zhou (2022) Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, Cited by: §I, §I-A. [25] A. Yang et al. (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §VII-G. [26] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, Cited by: §I-A. [27] L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, et al. (2023) Judging LLM-as-a-judge with MT-Bench and chatbot arena. In Advances in Neural Information Processing Systems, Cited by: §I-D. Appendix A Tool Specifications Table XII provides the complete tool reference. TABLE XII: Complete tool reference. Tool Parameters Description web_search query: str Search web fetch_url url: str Fetch URL create_outline sections: list Create outline revise_outline slide_idx, ... Edit outline generate_slide slide_idx, ... Generate slide edit_slide slide_idx, ... Edit slide set_theme theme: str Set theme get_slide_content idx: int Get slide HTML delete_slide idx: int Remove slide reorder_slides order: list Reorder slides duplicate_slide idx: int Copy slide insert_slide pos: int Insert slide review_deck (none) Review deck finalize (none) End episode Appendix B Theme Definitions Table XIII lists the visual theme color palettes. TABLE XIII: Visual theme color palettes. Theme Bg Text Accent Secondary default (255,255,255) (33,33,33) (41,98,255) (100,181,246) dark (30,30,30) (240,240,240) (0,200,83) (76,175,80) corporate (245,245,245) (44,62,80) (52,73,94) (149,165,166) creative (255,253,231) (33,33,33) (255,87,34) (255,167,38) tech (18,18,18) (224,224,224) (0,229,255) (29,233,182) Color intensity interpolation: colors=0.0 produces grayscale; colors=1.0 produces full vivid colors. Appendix C Sample Trajectories Example Brief: ⬇ "topic": "Series B Funding Pitch - AI-Powered Supply Chain Platform", "audience": "venture capitalists", "num_slides": 10, "confidence": 1.0, "content": "company": "ChainMind AI", "problem": "Supply chain disruptions cost $184B annually", "solution": "AI predicting disruptions 14 days ahead", "traction": "arr": "$4.2M", "growth": "312% YoY", "ask": "$25M at $100M pre-money" Trajectory Summary: 18 turns, 10 slides created, final quality 0.847, completed successfully. Appendix D Inverse Specification Prompt The following prompt is used for the inverse specification reward: ⬇ You are analyzing a slide deck presentation. Based ONLY on the slide content, predict what the original brief/requirements were. Return a JSON object with: "topic": "The main topic or title", "audience": "Who this targets", "num_slides": <intended count>, "key_themes": ["theme1", "theme2", "theme3"] Return ONLY the JSON object. No explanation. The reconstruction score is computed by comparing predicted values against the actual brief across four dimensions: topic similarity, audience match, slide count accuracy, and theme coverage. Manuscript received March 2026. Karthik Ragunath Ananda Kumar and Subrahmanyam Arunachalam contributed equally to this work.

Instant research discovery