Paper deep dive
From Trajectories to Instructions: Language-Conditioned Meta-Reinforcement Learning
Garvit Singla, Uma Maheswari Natarajan, Raghuram Bharadwaj Diddigi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/22/2026, 3:09:22 AM
Summary
The paper proposes LA-MAML (Language Adapted MAML), a meta-reinforcement learning framework that replaces the traditional gradient-based inner-loop adaptation of Model-Agnostic Meta-Learning (MAML) with a language-conditioned parameter offset. By using a pretrained Sentence Transformer to encode natural language task instructions into embeddings, LA-MAML directly modulates global policy parameters to adapt to new tasks without collecting trajectories for inner-loop updates. Experiments on the BabyAI benchmark demonstrate that LA-MAML achieves competitive or superior performance compared to baselines like standard MAML and ANIL, while significantly reducing per-iteration wall-clock training time.
Entities (8)
Relation Signals (7)
LA-MAML → evaluatedon → BabyAI
confidence 98% · Experiments on the BabyAI benchmark demonstrate that LA-MAML achieves competitive or improved performance
LA-MAML → replaces → gradient-based inner-loop adaptation
confidence 95% · LA-MAML takes a different approach by removing the inner loop entirely and performing adaptation through language-conditioned parameter modulation
LA-MAML → uses → Natural Language Instructions
confidence 95% · Leveraging these instructions as a direct task-specific signal, we propose LA-MAML
Sentence Transformer → usedby → LA-MAML
confidence 92% · In our approach, this encoder is a pretrained Sentence Transformer [15] model
LA-MAML → outperformsorcompeteswith → ANIL
confidence 90% · LA-MAML achieves competitive or improved performance compared to baselines... compared to other baselines.
LA-MAML → outperformsorcompeteswith → MAML
confidence 90% · LA-MAML achieves competitive or improved performance compared to baselines at a significantly lower per-iteration wall-clock training time.
LA-MAML → usesoptimizer → TRPO
confidence 88% · the outer loop applies a Trust Region Policy Optimization (TRPO) [17] based natural gradient update
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Model-Agnostic Meta-Learning (MAML) is a widely used framework for reinforcement learning (RL) that enables efficient transfer by learning global policy parameters that can be rapidly adapted to new tasks. MAML training proceeds in two loops: an inner loop where the global parameters are adapted to task-specific parameters, and an outer loop where these task-specific parameters are evaluated and losses are back-propagated to improve the global parameters. Traditionally, the inner loop adaptation is performed by collecting trajectories from the task environment and applying gradient updates on the empirical expected return, which can be a costly operation. We note that it is the outer loop that drives the actual learning of global parameters, and therefore the inner loop adaptation mechanism need not be restricted to be gradient-based. This observation leads us to ask: Can we replace the inner loop trajectory collection and gradient update with a simpler, task-specific signal? In many practical settings, tasks are naturally accompanied by language instructions. Leveraging these instructions as a direct task-specific signal, we propose LA-MAML (Language Adapted MAML), which modifies the inner loop by adapting the global policy parameters in a single step through a learned embedding of the task instruction, replacing the inner loop trajectory collection and gradient-based updates. Experiments on the BabyAI benchmark demonstrate that LA-MAML achieves competitive or improved performance compared to baselines at a significantly lower per-iteration wall-clock training time. These results demonstrate that language instructions are an effective and efficient substitute for trajectory-based inner loop adaptation in meta RL.
Tags
Links
- Source: https://arxiv.org/abs/2607.18830v1
- Canonical: https://arxiv.org/abs/2607.18830v1
Trouble viewing inline? Open PDF directly →
Full Text
33,646 characters extracted from source content.
Expand or collapse full text
11institutetext: International Institute of Information Technology Bangalore, India From Trajectories to Instructions: Language-Conditioned Meta-Reinforcement Learning Garvit Singla Corresponding author: garvit.singla@iiitb.ac.in Uma Maheswari Natarajan Raghuram Bharadwaj Diddigi Dr. Raghuram Bharadwaj is supported by the Anusandhan National Research Foundation (ANRF) under the Prime Minister Early Career Research Grant ANRF/ECRG/2024/005235/ENS Abstract Model-Agnostic Meta-Learning (MAML) is a widely used framework for reinforcement learning (RL) that enables efficient transfer by learning global policy parameters that can be rapidly adapted to new tasks. MAML training proceeds in two loops: an inner loop where the global parameters are adapted to task-specific parameters, and an outer loop where these task-specific parameters are evaluated and losses are back-propagated to improve the global parameters. Traditionally, the inner loop adaptation is performed by collecting trajectories from the task environment and applying gradient updates on the empirical expected return, which can be a costly operation. We note that it is the outer loop that drives the actual learning of global parameters, and therefore the inner loop adaptation mechanism need not be restricted to be gradient-based. This observation leads us to ask: Can we replace the inner loop trajectory collection and gradient update with a simpler, task-specific signal? In many practical settings, tasks are naturally accompanied by language instructions. Leveraging these instructions as a direct task-specific signal, we propose LA-MAML (Language Adapted MAML), which modifies the inner loop by adapting the global policy parameters in a single step through a learned embedding of the task instruction, replacing the inner loop trajectory collection and gradient-based updates. Experiments on the BabyAI benchmark demonstrate that LA-MAML achieves competitive or improved performance compared to baselines at a significantly lower per-iteration wall-clock training time. These results demonstrate that language instructions are an effective and efficient substitute for trajectory-based inner loop adaptation in meta RL. 1 Introduction Reinforcement Learning (RL) [21] provides a principled framework for training agents to learn sequences of actions that maximize long-term reward in uncertain environments. When combined with neural networks, deep reinforcement learning (deep RL) [11] has demonstrated success across a wide range of real-world domains, including autonomous driving [2], robotics [8, 7], healthcare [9], and strategic games [19, 12]. Despite these successes, deep RL methods are often computationally intensive and highly sensitive to hyperparameter choices, making training new tasks from scratch costly and brittle. Consequently, enabling agents to reuse knowledge acquired from prior tasks to accelerate learning on related tasks has become increasingly important. Transfer learning addresses this challenge by allowing RL agents to leverage previously learned representations and behaviors [16], thereby reducing data requirements and training time for new tasks. Such capability is particularly critical in practical settings where rapid adaptation is essential and repeated interaction with the environment is expensive or limited. Model-Agnostic Meta-Learning (MAML) [5] is a widely used transfer learning framework designed to enable rapid adaptation across tasks in both supervised learning and reinforcement learning settings. The central objective of MAML is to learn a global parameter initialization that can be easily adapted to related downstream tasks. In supervised learning, this adaptation is performed by minimizing the task-specific loss on the provided training examples, and the global parameters are subsequently optimized such that these adapted solutions perform well on held-out validation data. This process encourages the global parameters to lie in a region of the parameter space from which effective task adaptation is possible. During inference, the learned global parameters are adapted using a small number of examples from a previously unseen task to obtain a (possibly approximate) task-specific solution; even when not optimal, this adapted solution typically serves as a strong initialization. In the reinforcement learning setting, task adaptation proceeds differently. The global policy parameters are adapted to the specific tasks using policy gradient updates computed from trajectories collected by interacting with the environment. These trajectories are generated by executing the policy corresponding to the current global parameters. The global parameters are then optimized such that the adapted policies achieve high return on their respective tasks. This interaction-based adaptation introduces an inherent trade-off during inference. Using a small number of trajectories leads to high-variance gradient estimates and unstable adaptation, while collecting more trajectories improves stability at the cost of increased computational and interaction overhead. Motivated by the limitations of interaction-driven adaptation, we investigate whether a simpler and more reliable mechanism can be used to adapt global policy parameters to new tasks. In many practical reinforcement learning settings, tasks are naturally accompanied by high-level natural language descriptions that specify the task objectives. We leverage this structure and propose Language-Adapted MAML (LA-MAML), a meta-learning framework that uses task instructions as a consistent signal for adaptation. The core idea is to jointly train a shared global policy parameterization and a language encoder such that their combination yields a task-specific policy that is well aligned with the corresponding task. During inference, adaptation to a new task requires only computing the embedding of the task instruction (from the trained encoder) and combining it with the global parameters, resulting in an effective task-specific policy. An overview of this inference procedure is illustrated in Figure 1. Figure 1: Our Proposed LA-MAML trains the global parameters θ∗θ^* and trained encoder ϕ∗φ^* in an end-to-end fashion. During inference, the task adaption for task τ is obtained by simply adding θ∗θ^* and ϕ∗(τ)φ^*(τ) The key contributions of this work are summarized as follows: 1. We propose LA-MAML, a novel meta-learning framework for RL that leverages natural language task descriptions through a learned encoder–adapter mechanism for task-specific policy adaptation. 2. We empirically evaluate LA-MAML on the BabyAI benchmarks, showing faster convergence during training (wall-clock time per iteration) and improved performance during inference compared to other baselines. 3. We conduct ablation study to better understand the role of language in our proposed task adaptation. 2 Related Works In this section, we review prior literature by categorizing them into three key areas relevant to our work: efficiency in MAML, transfer learning extensions of MAML, and language-based task descriptions in meta-RL. 2.1 Efficiency in MAML In standard MAML, the framework incorporates two loops: an inner loop for adapting to individual tasks and an outer loop for generalization across a distribution of tasks. However, the outer-loop optimization requires differentiating through the inner-loop updates, introducing second-order gradients and increasing computational cost. Reptile [13] avoids this by using only first-order information by repeatedly sampling tasks, performing task-specific gradient updates, and moving the initialization toward the adapted weights of those tasks. Next, [14] investigates whether MAML’s effectiveness arises from rapid inner-loop adaptation or from feature reuse across tasks. The authors find that feature reuse is the dominant factor, leading to the development of ANIL (Almost No Inner Loop), which updates only the task-specific head while keeping the shared backbone fixed. In [23], the CAVIA method partitions model parameters into global shared weights and low-dimensional context parameters, of which only the latter are updated per task. While methods such as ANIL, CAVIA, and WarpGrad [6] explore ways to simplify or stabilize the inner loop by restricting adaptation to small parameter subsets, introducing context vectors, or re-parameterizing the gradient space, they continue to operate within the confines of gradient-based meta-optimization. In contrast, LA-MAML takes a different approach by removing the inner loop entirely and performing adaptation through language-conditioned parameter modulation, thereby reducing computational overhead. 2.2 Transfer Learning Extensions of MAML While early variants of MAML primarily focused on improving computational efficiency within the inner loop, a complementary research direction has explored enhancing MAML’s transferability across tasks. MetaICL [10] introduces a meta-training framework where large pretrained language models (LMs) are fine-tuned on a diverse collection of tasks to improve in-context learning. The work by [4] explores whether meta-learning can enhance multi-task instructional learning (MTIL), where LMs are trained to follow natural language instructions across tasks. These studies extend MAML into the domain of large-scale language modeling, showing that meta-training across diverse tasks enhances in-context generalization. However, they still depend on gradient-based or data-driven meta optimization, where task adaptation is achieved implicitly through context or learned initialization. 2.3 Language-Based Task Descriptions in Meta-RL Finally, we highlight works showing how language can serve as a structured channel for task specification in meta-RL. The authors in [1] propose a meta-RL algorithm that uses natural language instructions with reward signals to guide adaptation across multiple manipulation tasks. The work by [22] introduces a dual-MDP formulation that incorporates both task symmetry and language instructions into meta-RL. Together with the imitation learning framework in [20], these studies show that natural language provides an interpretable representation of task intent for guiding policy adaptation and improving generalization. Building on this foundation, our proposed LA-MAML extends the integration of language by treating language embeddings as direct modulators of model parameters, enabling more efficient and interpretable task adaptation. 3 Background We first briefly describe the Model-Agnostic Meta-Learning (MAML) algorithm [5], followed by our proposed approach. MAML is a widely used meta-learning framework designed to enable rapid adaptation to new tasks without retraining from scratch. It achieves this by learning an initialization of parameters θ, that can be efficiently adapted to new tasks using only a few task-specific samples. To achieve this, MAML operates in two loops: inner-loop (task-specific adaptation) and outer loop (meta-update of global parameters). To formalize this process, tasks are sampled from a distribution p()p(T) [5], which defines the task space considered during meta-training: T1,T2,…,Tn∼p(),T_1,T_2,…,T_n p(T), (1) where each task TiT_i is associated with its own dataset. The model fθf_θ, parameterized by θ, corresponds to a policy πθ(a|s) _θ(a|s) in reinforcement learning. For each task TiT_i, trajectories are collected using πθ _θ , which are used to compute the task-specific loss LTi(πθ)L_T_i( _θ), defined as the negative expected return. The parameters are then adapted via gradient descent: θi′=θ−α∇θLTi(πθ),θ _i=θ-α _θL_T_i( _θ), (2) where α is the inner-loop learning rate. After task-specific adaptation, the outer loop optimizes the initialization θ to generalize across tasks. The adapted parameters θi′θ _i define the policy πθi′ _θ _i, whose performance on task TiT_i yields the loss LTi(πθi′)L_T_i( _θ _i). Aggregating these losses across tasks gives the meta-objective: ℒmeta(θ)=∑Ti∼p()LTi(πθi′),L_meta(θ)= _T_i p(T)L_T_i\! ( _θ _i ), (3) which is used to update θ: θ←θ−β∇θℒmeta(θ),θ←θ-β _θL_meta(θ), (4) where β is the meta-learning rate. This enables the model to achieve strong performance on new tasks with minimal adaptation. However, MAML relies on gradient-based adaptation requiring repeated trajectory collection, which becomes challenging in stochastic environments. Limited trajectories lead to high-variance gradient estimates and unstable policies, while increasing trajectories improves stability at the cost of higher interaction and computational overhead. This challenge persists both during meta-training and at inference, where adapting to new tasks requires environment interaction. To address these limitations, we propose a framework that leverages natural language instructions as a structured representation of task objectives. By mapping language descriptions to parameter adjustments, our approach enables direct and efficient task adaptation without requiring trajectory collection or gradient-based inner-loop updates during adaptation. This reduces the computational burden associated with MAML while providing a more flexible mechanism for task adaptation. 4 Proposed Methodology We propose a novel meta-learning framework that accelerates task adaptation through a language-based setup, where task instructions are expressed in natural language and transformed into feature vectors. This approach bypasses the need for gradient-driven updates, which lie at the core of the conventional gradient-based procedure of MAML [5]. In our formulation, the adapted parameters θi′θ _i, which define the policy πθi′ _θ _i is obtained without explicit trajectory collection or gradient steps. Rather, they are derived through a direct offset, derived from natural language task descriptions: θi′=θ+δfϕ(task),θ _i=θ+δ f_φ(task), (5) where δ is the step-size parameter, fϕ(task)f_φ(task) is a learned function, parameterized by ϕφ, that maps the natural language description of a task into task-specific parameter offsets. To realize this mapping function fϕf_φ, two integrated components are used. The first is a task encoder which transforms the natural language description into an embedding that captures the semantic intent of the task. In our approach, this encoder is a pretrained Sentence Transformer [15] model (all-MiniLM-L6-v2). The second is an adapter network that projects this embedding into the policy parameter space. It generates task-specific offsets that convert the high-level task representation into concrete parameter updates. These updates are then combined with the global initialization θ to yield the adapted parameters θi′θ _i. While our approach introduces novelty by replacing the gradient-based inner-loop with direct language driven parameter adaptation, the outer loop continues to function analogously to MAML. It optimizes the global initialization θ so that it remains broadly adaptable across tasks, while simultaneously training the language parameters ϕφ of adapter network to generate effective task-specific offsets from the frozen language embeddings. Through this joint optimization, the model not only preserves a generalizable initialization but also learns to exploit task descriptions for rapid adaptation. Therefore, the meta-objective for the proposed framework is defined as: ℒmeta(θ,ϕ)=∑Ti∼p()LTi(πθ+δfϕ(Ti)),L_meta(θ,φ)= _T_i p(T)L_T_i\! ( _θ+δ f_φ(T_i) ), (6) where each loss LTiL_T_i is computed on trajectories sampled from the adapted policy πθ+δfϕ(Ti) _θ+δ f_φ(T_i), which results from merging the global initialization with the task-specific adjustment. To realize this meta-objective, the outer loop applies a Trust Region Policy Optimization (TRPO) [17] based natural gradient update to the joint parameters [θ,ϕ][θ,φ]: [θ,ϕ]←[θ,ϕ]−∇~θ,ϕℒmeta(θ,ϕ),[θ,φ]\;←\;[θ,φ]\;-\; ∇_θ,φ\,L_meta(θ,φ), (7) where ∇~ ∇ denotes the natural gradient computed under the TRPO constraint. In this proposed framework, the model learns a broadly adaptable global policy initialization parameters θ and an effective task-encoding function fϕf_φ, enabling efficient adaptation to unseen tasks without explicit trajectory collection or gradient updates in inner-loop. During inference, the task-specific parameters of the new task TnewT_new is obtained as: θnew′=θ+δfϕ(Tnew)θ _new=θ+δ f_φ(T_new). This allows the model to generalize quickly to novel tasks with minimal computational overhead. We present the complete pseudo-code of our proposed approach in Algorithm 1. Algorithm 1 Language Adapted Policy for MAML (LA-MAML) 1:Task distribution p()p(T), hyperparameters (episodes K, weighting factor λ, discount factor γ, learning rate δ) 2:Randomly initialize policy parameters θ and task-encoding parameters ϕφ 3:Initialize frozen pretrained Sentence Transformer encoder 4:for meta-iteration =1=1 to N do 5: Sample set of tasks Ti∼p()T_i p(T) 6: for each task TiT_i do 7: sis_i ← natural-language instruction for TiT_i 8: task embedding ei←e_i← encode sis_i into a dense 9: semantic representation using the 10: Sentence Transformer encoder 11: parameter offsets f~i f_i ← project eie_i into policy 12: parameter space θ using the adapter 13: network 14: task-specific parameters fϕ(Ti)f_φ(T_i) ← reshape f~i f_i to 15: match the layer dimensions of θ′sθ s 16: Obtain task-adapted parameters θi′θ _i with no 17: inner-loop gradients 18: θi′θ _i ← θ + δfϕ(Ti)δ f_φ(T_i) 19: Collect K trajectories using policy πθi′ _θ _i and store them 20: in buffer space iD_i 21: Fit value function VψV_ψ on trajectories in iD_i 22: Compute temporal-difference residuals δt _t using VψV_ψ 23: δt=rt+γVψ(st+1)−Vψ(st) _t=r_t+γ V_ψ(s_t+1)-V_ψ(s_t) 24: Compute advantages A^t A_t using GAE[18]: 25: A^t=δt+γλA^t+1 A_t= _t+γλ A_t+1 26: Compute task-specific loss LTiL_T_i 27: LTi←−(s,a)∼i[logπ(a∣s;θi′)A^t]L_T_i←-E_(s,a) _i [ π(a s;θ _i)\, A_t ] 28: Compute Meta loss 29: ℒmeta(θ,ϕ)←∑iLTiL_meta(θ,φ)← _iL_T_i 30: Update global parameters using TRPO[17] 31: (θ,ϕ)←TRPO((θ,ϕ);ℒmeta(θ,ϕ))(θ,φ) ((θ,φ);\ L_meta(θ,φ)) 32:return (θ,ϕ)(θ,φ) Through the proposed framework, we demonstrate that a language trained model can leverage human-interpretable task descriptions, enabling flexible adaptation to tasks specified in natural language. Together, these advantages lead to a more efficient and versatile transfer approach. 5 Experiments and Results We now begin by discussing the experiments and results obtained using our proposed LA-MAML framework, as outlined in the Proposed Methodology section 4. The objective is to evaluate the effectiveness of language-driven adaptation in diverse task settings without relying on gradient-based inner-loop updates. To this end, we adopt the BabyAI framework [3], a MiniGrid-based grid world where tasks are specified in natural language. The BabyAI environment comprises of connected rooms populated with multiple distractor objects where the agent must navigate and interact with objects to execute the given instruction correctly. We evaluate our method on seven BabyAI environments: GoToLocal, PickupDist, GoToObjDoor, GoToOpen, OpenDoor, OpenDoorLoc, and OpenDoorsOrder, covering a broad range of task compositions and difficulty levels. BabyAI features sparse rewards setting where the agent receives a reward only upon successful task completion, with no intermediate feedback. The following subsections describe the experimental setup and results, and analyze the contribution of individual components of our proposed algorithm through an ablation study. 5.1 Experimental Setup Each BabyAI environment exhibits distinct dynamics and can be instantiated in multiple configurations that vary in room size (S) and number of distractors (N). For example, GoToLocal includes configurations such as GoToLocalS5N2, GoToLocalS7N4, and GoToLocalS8N3 which progressively increase task complexity. For a given configuration, tasks are specified by language-based goals over target objects. Using this setup, for each environment we sample a set of training tasks while reserving a separate set of unseen tasks for evaluation. During evaluation, we test on the held-out tasks across multiple configurations. For comparison, we consider three baselines: (i) standard MAML [5], which performs gradient-based meta-learning across tasks, (i) a language-conditioned policy without meta-learning, which maps the state and task instruction (s,ℓ)(s, ) directly to actions, similar to BabyAI framework [3] and does not perform inner or outer-loop updates, and (i) ANIL (Almost No Inner Loop) [14], a simplified variant of MAML where inner-loop adaptation updates only the task-specific head (final layer) while keeping the shared representation network fixed. Further, all methods share the same environment observation space. To ensure a fair comparison, we use identical network architectures and hyperparameters across all comparison methods. 5.2 Results and Discussion Figure 2: Training curves shows the convergence over meta-iterations for LA-MAML and baselines across seven environments. The shaded regions correspond to the standard deviation, while the solid lines represent the mean average steps per meta-iteration. Table 1: Time taken per iteration during training (in seconds). Environment LA-MAML MAML Language cond. Policy ANIL GoToLocal 5.31 7.42 2.53 7.00 PickupDist 8.04 12.47 19.01 21.21 GoToObjDoor 6.63 13.17 7.51 12.93 GoToOpen 26.04 40.73 22.19 26.85 OpenDoor 10.02 25.02 13.27 17.01 OpenDoorLoc 13.39 24.94 24.43 19.82 OpenDoorsOrder 19.91 39.73 35.86 33.23 Table 2: Performance comparison of LA-MAML with standard MAML, Language-conditioned Policy and ANIL across environments on new tasks using average steps (Mean ± Standard Deviation), including both successful and failed episodes, where maximum steps are capped as per environment dynamics (Lower is better). Environment LA-MAML MAML Language conditioned Policy ANIL GoToLocal 43.59±22.3543.59± 22.35 44.19±17.2744.19± 17.27 73.18±28.0573.18± 28.05 47.43±20.4147.43± 20.41 PickupDist 194.15±62.03194.15± 62.03 286.69±78.11286.69± 78.11 358.39±68.66358.39± 68.66 256.12±59.97256.12± 59.97 GoToObjDoor 54.86±22.6554.86± 22.65 64.40±28.2264.40± 28.22 87.55±39.5487.55± 39.54 65.35±30.3965.35± 30.39 GoToOpen 579.48±149.74579.48± 149.74 638.07±137.86638.07± 137.86 659.26±147.80659.26± 147.80 635.25±142.82635.25± 142.82 OpenDoor 204.28±74.65204.28± 74.65 560.73±100.61560.73± 100.61 463.42±90.21463.42± 90.21 561.06±95.53561.06± 95.53 OpenDoorLoc 135.58±91.36135.58± 91.36 516.08±115.41516.08± 115.41 418.86±110.11418.86± 110.11 487.98±117.36487.98± 117.36 OpenDoorsOrder 317.59±172.37317.59± 172.37 553.73±272.87553.73± 272.87 568.50±267.76568.50± 267.76 514.78±251.16514.78± 251.16 Fig. 2 shows that LA-MAML consistently converges to lower average steps and at a faster rate compared to MAML, ANIL, and the language-conditioned policy baseline during the training. This demonstrates that replacing the gradient-based inner loop with direct language adaptation leads to more efficient training. Further, from Tables 1, 2, we infer that: 1. LA-MAML achieves better evaluation performance while reducing wall-clock training time compared to gradient-based meta-learning baselines such as MAML and ANIL in most settings. Although it is not always the fastest method per iteration among all baselines, it provides a stronger balance between computational cost and adaptation performance. 2. The language-conditioned policy underperforms compared to LA-MAML on downstream tasks, indicating that language information alone is insufficient for effective adaptation. Hence, improved performance of LA-MAML highlights the importance of meta-learning in enabling robust generalization across tasks. Overall, these results suggest that replacing the inner loop gradient update with a single step language-conditioned adaptation not only reduces per-iteration wall-clock training time but also yields competitive or improved task performance across environments 111The source code, hyperparameter settings and additional experimental results are available at: https://github.com/garvitsingla/LA-MAML 5.3 Ablation Study We conduct an ablation study to further analyze the performance of our proposed approach. Table 3: Comparison of LA-MAML against inference with global parameters θ only. (Lower is better) Environment LA-MAML Inference using θ only GoToLocal 44.32±26.4944.32± 26.49 59.27±24.0559.27± 24.05 PickupDist 202.48±54.99202.48± 54.99 263.88±49.16263.88± 49.16 GoToObjDoor 59.84±28.8659.84± 28.86 83.38±40.8283.38± 40.82 GoToOpen 576.11±132.14576.11± 132.14 658.25±174.74658.25± 174.74 OpenDoor 193.90±60.35193.90± 60.35 450.68±101.08450.68± 101.08 OpenDoorLoc 141.93±96.33141.93± 96.33 435.60±133.08435.60± 133.08 OpenDoorsOrder 299.52±179.98299.52± 179.98 620.69±258.27620.69± 258.27 In this experiment, we analyze the impact of language during inference in the LA-MAML framework. The standard evaluation in LA-MAML uses both the learned global policy parameters θ and the language parameters ϕφ. Therefore during inference, we perform an ablation in which only the optimized global policy parameters are used, i.e., θ′=θ =θ so that the agent executes the task without relying on ϕφ. As shown in Table 3, removing the language component results in a performance drop, indicating that the language parameters capture essential task-specific information during training that is later transferred and utilized at evaluation. This highlights that both θ and ϕφ contribute significantly to the model’s learning and execution: θ provides transferable knowledge, while ϕφ imbues the policy with semantic context that guides task-specific adaptation. 5.4 Limitations LA-MAML shows promising results in BabyAI settings, where templated instructions provide reliable task-relevant signals about the goal object, color, or navigation objective. However, evaluating LA-MAML with more natural and potentially imperfect language, where task descriptions may be less structured, incomplete, or noisy, remains an important future direction. 6 Conclusion In this work, we propose LA-MAML, a novel meta-reinforcement learning framework that leverages natural language as task descriptors to enable efficient and interpretable task adaptation. Unlike standard MAML, which depends on gradient-based inner-loop updates and trajectory collection for each new task, LA-MAML performs direct parameter adaptation driven by language semantics. This allows the model to infer task-specific behavior directly from linguistic cues, leading to faster and more efficient adaptation. The experiments across seven benchmark BabyAI environments demonstrate that language-driven adaptation achieves strong performance and computationally more efficient compared to both gradient-based meta-learning and a language-conditioned policy. References [1] Z. Bing, A. Koch, X. Yao, K. Huang, and A. Knoll (2022) Meta-reinforcement learning via language instructions. arXiv preprint arXiv:2209.04924. Cited by: §2.3. [2] Bojarski, Mariusz and Del Testa, Davide and Dworakowski, Daniel and Firner, Bernhard and Flepp, Beat and Goyal, Prasoon and Jackel, Lawrence D and Monfort, Mathew and Muller, Urs and Zhang, Jiakai and others (2016) End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316. Cited by: §1. [3] M. Chevalier-Boisvert, D. Bahdanau, S. Lahlou, L. Willems, C. Saharia, T. H. Nguyen, and Y. Bengio (2018) Babyai: a platform to study the sample efficiency of grounded language learning. arXiv preprint arXiv:1810.08272. Cited by: §5.1, §5. [4] B. Deb, A. Hassan, and G. Zheng (2022) Boosting natural language generation from instructions with meta-learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 6792–6808. Cited by: §2.2. [5] C. Finn, P. Abbeel, and S. Levine (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, p. 1126–1135. Cited by: §1, §3, §3, §4, §5.1. [6] S. Flennerhag, A. A. Rusu, R. Pascanu, F. Visin, H. Yin, and R. Hadsell (2019) Meta-learning with warped gradient descent. arXiv preprint arXiv:1909.00025. Cited by: §2.1. [7] J. Ibarz, J. Tan, C. Finn, M. Kalakrishnan, P. Pastor, and S. Levine (2021) How to train your robot with deep reinforcement learning: lessons we have learned. The International Journal of Robotics Research 40 (4-5), p. 698–721. Cited by: §1. [8] S. Levine, C. Finn, T. Darrell, and P. Abbeel (2016) End-to-end training of deep visuomotor policies. Journal of Machine Learning Research 17 (39), p. 1–40. Cited by: §1. [9] M. Liu, X. Shen, and W. Pan (2022) Deep reinforcement learning for personalized treatment recommendation. Statistics in medicine 41 (20), p. 4034–4056. Cited by: §1. [10] S. Min, M. Lewis, L. Zettlemoyer, and H. Hajishirzi (2022) MetaICL: learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Cited by: §2.2. [11] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. Cited by: §1. [12] Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Rusu, Andrei A and Veness, Joel and Bellemare, Marc G and Graves, Alex and Riedmiller, Martin and Fidjeland, Andreas K and Ostrovski, Georg and others (2015) Human-level control through deep reinforcement learning. Nature 518, p. 529–533. Cited by: §1. [13] A. Nichol, J. Achiam, and J. Schulman (2018) On first-order meta-learning algorithms. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §2.1. [14] A. Raghu, M. Raghu, S. Bengio, and O. Vinyals (2019) Rapid learning or feature reuse? towards understanding the effectiveness of maml. arXiv preprint arXiv:1909.09157. Cited by: §2.1, §5.1. [15] N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 3982–3992. Cited by: §4. [16] A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osindero, and R. Hadsell (2018) Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960. Cited by: §1. [17] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz (2015) Trust region policy optimization. In International conference on machine learning, p. 1889–1897. Cited by: §4, 30. [18] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel (2015) High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438. Cited by: 24. [19] Silver, David and Huang, Aja and Maddison, Chris J and Guez, Arthur and Sifre, Laurent and Van Den Driessche, George and Schrittwieser, Julian and Antonoglou, Ioannis and Panneershelvam, Veda and Lanctot, Marc and others (2016) Mastering the game of go with deep neural networks and tree search. Nature 529, p. 484–489. Cited by: §1. [20] S. Stepputtis, J. Campbell, M. Phielipp, S. Lee, C. Baral, and H. Ben Amor (2020) Language-conditioned imitation learning for robot manipulation tasks. In Proceedings of the 2020 Conference on Robot Learning (CoRL), Cited by: §2.3. [21] R. S. Sutton and A. G. Barto (2018) Reinforcement learning: an introduction. 2 edition, MIT Press. Cited by: §1. [22] X. Yao, Z. Bing, G. Zhuang, K. Chen, H. Zhou, K. Huang, and A. Knoll (2022) Learning from symmetry: meta-reinforcement learning with symmetrical behaviors and language instructions. In Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Cited by: §2.3. [23] L. Zintgraf, K. Shiarlis, V. Kurin, K. Hofmann, and S. Whiteson (2019) Fast context adaptation via meta-learning. In Proceedings of the 36th International Conference on Machine Learning (ICML), Cited by: §2.1.