Paper deep dive
Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping
Vaishnav Vaidheeswaran, Dilith Jayakody, Biruk Ambaw, Jaswanth Kumar, Md Mahbub Alam, Gabriel Spadon
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/9/2026, 1:08:49 AM
Summary
This study evaluates the utility of latent context in Inverse Reinforcement Learning (IRL) for Arctic shipping navigation. Using 3,186 AIS-derived voyages from 202 vessels, the authors compare linear shared reward (MCE-IRL), nonlinear shared reward (AIRL), and latent-context models (PEMIRL). Results indicate that nonlinear reward capacity significantly improves predictive accuracy and route fidelity compared to linear models, while adding per-vessel latent context actually reduces performance by 16.5%. The findings suggest that behavioral heterogeneity in Arctic shipping is largely explained by observable route and environmental conditions rather than hidden vessel-specific preferences, challenging the necessity of latent context in this domain.
Entities (8)
Relation Signals (5)
Nonlinear Reward → improves → Held-out Likelihood
confidence 95% · The nonlinear reward improves held-out likelihood by 50.9% over the linear baseline.
AIRL → outperforms → MCE-IRL
confidence 95% · The nonlinear reward improves held-out likelihood by 50.9% over the linear baseline.
PEMIRL → underperforms → AIRL
confidence 95% · Adding vessel-specific latent context reduces performance by 16.5%.
Latent Context → doesnotimprove → Predictive Accuracy
confidence 92% · Adding vessel-specific latent context reduces performance by 16.5%.
Observable Features → explains → Behavioral Variation
confidence 90% · Apparent vessel-level variation is largely explained by observable route and environmental conditions rather than hidden vessel-specific factors.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Artificial Intelligence (AI)-assisted navigation can help Arctic shipping adapt to rapidly changing sea-ice conditions, but reliable deployment requires reward models that are interpretable and robust to changing environments. Inverse reinforcement learning (IRL) provides a framework for recovering such rewards from vessel trajectories, while recent meta-IRL methods introduce latent context variables to capture behavioral heterogeneity. However, it remains unclear whether these latent representations recover genuinely hidden preferences or simply re-encode information already available in the observed state. We conduct a controlled evaluation on 3,186 AIS-derived voyages from 202 vessels across nine Arctic shipping seasons, comparing a linear shared reward, a nonlinear shared reward, and a latent-context model built on the same nonlinear architecture. The nonlinear reward improves held-out likelihood by 50.9% over the linear baseline, whereas adding vessel-specific latent context reduces performance by 16.5%. Behavioral analysis, context probes, and a pre-registered feature-hiding ablation show that apparent vessel-level variation is largely explained by observable route and environmental conditions rather than hidden vessel-specific factors. Moreover, predictive accuracy, route fidelity, and reward transfer yield different model rankings, demonstrating that no single metric is sufficient to evaluate learned rewards. These findings motivate testing whether the observed route, environmental, and vessel features already explain behavioral variation before adding per-vessel latent context. This supports more trustworthy AI deployment in safety-critical domains.
Tags
Links
- Source: https://arxiv.org/abs/2608.06105v1
- Canonical: https://arxiv.org/abs/2608.06105v1
Trouble viewing inline? Open PDF directly →
Full Text
77,432 characters extracted from source content.
Expand or collapse full text
Does Latent Context Help? A Controlled Evaluation of Inverse Reinforcement Learning in Arctic Shipping Vaishnav Vaidheeswaran,1 Dilith Jayakody,1 Biruk Ambaw,1 Jaswanth Kumar,2 Md Mahbub Alam,1 Gabriel Spadon1,* Abstract Artificial Intelligence (AI)-assisted navigation can help Arctic shipping adapt to rapidly changing sea-ice conditions, but reliable deployment requires reward models that are interpretable and robust to changing environments. Inverse reinforcement learning (IRL) provides a framework for recovering such rewards from vessel trajectories, while recent meta-IRL methods introduce latent context variables to capture behavioral heterogeneity. However, it remains unclear whether these latent representations recover genuinely hidden preferences or simply re-encode information already available in the observed state. We conduct a controlled evaluation on 3,186 AIS-derived voyages from 202 vessels across nine Arctic shipping seasons, comparing a linear shared reward, a nonlinear shared reward, and a latent-context model built on the same nonlinear architecture. The nonlinear reward improves held-out likelihood by 50.9% over the linear baseline, whereas adding vessel-specific latent context reduces performance by 16.5%. Behavioral analysis, context probes, and a pre-registered feature-hiding ablation show that apparent vessel-level variation is largely explained by observable route and environmental conditions rather than hidden vessel-specific factors. Moreover, predictive accuracy, route fidelity, and reward transfer yield different model rankings, demonstrating that no single metric is sufficient to evaluate learned rewards. These findings motivate testing whether the observed route, environmental, and vessel features already explain behavioral variation before adding per-vessel latent context. This supports more trustworthy AI deployment in safety-critical domains. Introduction Arctic sea ice is retreating faster than navigational experience is being acquired. Shipping seasons are lengthening, corridors that were impassable a decade ago now carry commercial traffic, and an ice regime that made a route safe in one season can make the same route hazardous in the next (Shu et al. 2026). Such expertise is difficult to transfer because it is held by a small number of ice navigators, depends on changing conditions, and is not recorded in a readily reusable form. Learning from demonstrations provides a way to capture such expertise, but the representation learned from demonstrations is critical. Behavioral cloning can reproduce historical decisions but may fail when conditions change, whereas inverse reinforcement learning (IRL) recovers the underlying objectives that guide decisions and allows behavior to be re-optimized under new conditions (Ziebart et al. 2008, 2010). Because Arctic sea-navigation conditions vary across seasons and years, transferable reward models provide a more adaptable alternative to memorizing past behavior. Real-world demonstrations, however, rarely come from identical decision makers. Vessels differ in capability, cargo, schedule pressure, and risk tolerance. Meta-IRL methods address this heterogeneity by introducing latent context variables that condition rewards on demonstrator-specific preferences (Yu et al. 2019; Li et al. 2017; Xu et al. 2019). While this assumption is well motivated, it has primarily been studied on synthetic benchmarks where task variation is predefined. In real environments, behavioral differences may instead arise from hidden preferences, observable conditions, or both. Thus, behavioral variation alone does not establish the need for latent context. This question is important for safety-relevant maritime applications, where learned behavioral models can support route analysis, situational awareness, and decision support under changing environmental conditions. Our goal is not to deploy autonomous navigation, but to evaluate whether learned rewards provide reliable representations of observed behavior. We investigate this question through a controlled study of nine seasons of Arctic Automatic Identification System (AIS) traffic, comprising 3,1863,186 voyages from 202202 cargo and tanker vessels. We represent navigation as a hexagonal graph Markov decision process (MDP) and examine whether vessel-level behavioral variation requires latent context or can be explained by observable features. We compare three reward models that isolate reward flexibility from latent entity-specific adaptation (hereafter, personalization): MCE-IRL (Ziebart et al. 2010), which learns a shared linear reward; AIRL (Fu et al. 2018), which introduces nonlinear shared rewards; and PEMIRL (Yu et al. 2019), which adds per-vessel latent context while retaining nonlinear rewards. Here, personalization refers to learning latent vessel-specific representations that adapt decision-making behavior based on information not captured by the observable state, rather than user preference modeling or recommendation. We make three contributions. First, we provide a controlled empirical evaluation of latent-context IRL on large-scale real-world trajectories, separating the effects of reward flexibility from latent context. Second, we develop analyses that distinguish behavioral variation explained by observable route, environmental, and vessel factors from unexplained variation. Third, we introduce a context-need diagnostic for determining when per-vessel latent context is likely to add decision-relevant information beyond the observed state. Our findings show that behavioral variation alone is insufficient evidence for latent context and motivate more careful evaluation of entity-specific adaptive AI systems. Related Work Classical maximum-entropy inverse reinforcement learning (MaxEnt IRL) learns reward functions from expert demonstrations under a maximum-entropy objective (Ziebart et al. 2008, 2010). In our experiments, maximum causal entropy IRL (MCE-IRL) represents one shared reward as a linear combination of observed features. Later methods introduced nonlinear reward representations. Guided Cost Learning estimates a neural cost using sampled trajectories (Finn et al. 2016), while Generative Adversarial Imitation Learning (GAIL) learns a policy through a discriminator that distinguishes expert from generated behavior (Ho and Ermon 2016; Osa et al. 2018). Adversarial Inverse Reinforcement Learning (AIRL) (Fu et al. 2018) adapts this framework for reward recovery by separating the underlying task objective from a potential-based shaping term, to improve transfer beyond training dynamics. We use MCE-IRL and AIRL as shared-reward models: both assume that one reward explains all demonstrators, but AIRL can represent nonlinear interactions among observable route, vessel, and environmental features. When demonstrations may arise from different tasks, goals, or behavioral modes, one approach is to infer a latent context from trajectories and condition the model on it. InfoGAIL (Li et al. 2017) uses discrete latent codes to identify imitation styles, whereas Probabilistic Embeddings for Meta-IRL (PEMIRL) (Yu et al. 2019) infers a distribution over task contexts and conditions both the reward f(s,a,z)f(s,a,z) and policy on the latent variable z. PEMIRL uses an information-maximization objective, similar to InfoGAN (Chen et al. 2016), to prevent the model from ignoring the latent context. Other meta-IRL approaches use gradient-based meta-learning to adapt reward parameters from a small number of demonstrations (Finn et al. 2017; Xu et al. 2019). Probabilistic context inference is also used in meta-RL, where recent experience identifies the current task and conditions the policy (Rakelly et al. 2019). These methods are commonly evaluated on benchmarks where meaningful task differences are present by construction. Such benchmarks test whether a model can recover latent variation known to exist, but not whether a latent variable is needed for naturally occurring behavioral differences. In real-world data, apparent differences may arise from hidden preferences or from observable environmental conditions, route assignments, and physical characteristics. We therefore treat vessel identity as a candidate source of latent context, rather than evidence that vessels optimize different hidden objectives, and ask whether vessel-specific variation remains after observable route, sea-ice, environmental, and vessel features are considered. This mirrors a long-standing distinction in econometric and transportation modeling between heterogeneity captured by observed covariates and genuinely unobserved heterogeneity requiring random or latent parameters (Heckman 1981; Mannering et al. 2016); the latent context in Eq. 3 is the IRL analogue of a random-parameters term, and the question is whether the data warrant one. Accurate prediction of held-out expert actions does not necessarily imply that a learned reward is useful to optimize. An imperfect proxy may fit demonstrations while assigning high reward to unintended behavior that a new agent can exploit (Amodei et al. 2016; Skalse et al. 2022). Studies of adversarial imitation learning also show that implementation, regularization, normalization, demonstration quality, and optimization choices can affect performance as much as the nominal algorithm (Orsini et al. 2021). We therefore hold data, features, splits, optimization budget, and random seeds fixed while changing one modeling factor at a time. Each reward is evaluated by its held-out prediction, complete-route fidelity, and ability to train a newly initialized policy. Related navigation work has produced several models. Learned costs support personalized A* route recommendation (Wang et al. 2019), while trajectory-prediction and mining methods forecast movement or identify mobility patterns (Nguyen and Fablet 2024; Spadon et al. 2025; Zheng 2015). In maritime settings, prior studies have optimized routes using fuel and risk objectives, trained goal-conditioned policies with hand-designed rewards, and assessed operations under changing sea-ice conditions (Zhang et al. 2022; Vaidheeswaran et al. 2025; Shu et al. 2026). These studies predict movement, characterize traffic, optimize routes, learn policies, or assess risk. To the best of our knowledge, prior work has not isolated whether a reward recovered from vessel trajectories benefits from vessel-specific latent context after controlling for nonlinear model capacity and observable route and environmental information. Methodology Preliminaries We work in a deterministic, goal-conditioned MDP ℳ=(,,T,ϕ,M,γ,H)M=(S,A,T,φ,M,γ,H), where states are navigable cells, T(s,a)T(s,a) is the deterministic successor, and ϕ(s)∈ℝDφ(s) ^D is a shared feature vector. A validity mask M(s,a)M(s,a) marks which actions exist at s; invalid actions receive zero probability in every policy, likelihood, chance baseline, and decoded route. A demonstration τ=(s0,a0,…,sL)τ=(s_0,a_0,…,s_L) is one voyage with demonstrator index v and absorbing goal g. MCE-IRL (Ziebart et al. 2010) assumes a linear reward rθ(s)=θ⊤ϕ(s)r_θ(s)=θ φ(s) and the maximum-causal-entropy policy it induces. Because our graph diameter (≈250≈ 250 cells) requires a finite horizon, soft value iteration proceeds for t=H−1,…,0t=H-1,…,0 as Qt(s,a) Q_t(s,a) =rθ(s)+Vt+1(T(s,a)), =r_θ(s)+V_t+1(T(s,a)), (1) Vt(s) V_t(s) =log∑a∈(s)exp(Qt(s,a)). = \! _a (s) (Q_t(s,a)). with Vt(g)=0V_t(g)=0 and logπt(a∣s)=Qt(s,a)−Vt(s) _t(a s)=Q_t(s,a)-V_t(s). Maximizing demonstration log-likelihood is convex in θ, with gradient equal to the difference between expert and model feature expectations. The resulting policy is non-stationary because it depends on the remaining horizon. AIRL (Fu et al. 2018) replaces the linear reward with a learned nonlinear reward. In the PEMIRL implementation, the discriminator predicts expert transitions and its logit f(s,a)f(s,a) is used directly as the reward; the generator is trained by PPO (Schulman et al. 2017). Canonical AIRL instead uses D(s,a,s′)=exp(f(s,a,s′))exp(f(s,a,s′))+π(a∣s),D(s,a,s )= (f(s,a,s )) (f(s,a,s ))+π(a s), with f(s,s′)=g(s)+γh(s′)−h(s),f(s,s )=g(s)+γ h(s )-h(s), (2) where the state-only g is the transferable reward and h is a shaping term. We evaluate both formulations. PEMIRL (Yu et al. 2019) extends this framework by conditioning both reward and policy on a latent context. A posterior q(z∣τ1:k)q(z τ^1:k) infers z∈ℝdzz ^d_z from a demonstrator’s support trajectories, the reward becomes f(s,a,z)f(s,a,z), and the policy becomes π(a∣s,z)π(a s,z). To prevent the latent context from being ignored, PEMIRL adds a reward-weighted information-maximization term inspired by InfoGAN (Chen et al. 2016): ℒinfo=−η[logq(z∣τ)(∑tf(st,at,z)−f¯)],L_info=-η\,E\! [ q(z τ) ( _tf(s_t,a_t,z)- f ) ], (3) where f¯ f is the mean trajectory reward within a rollout group. AIRL is the z-free special case of the same model, making the pair a clean control: only the context pathway differs. Reporting only MCE-IRL against PEMIRL, the usual evaluation, confounds nonlinear reward capacity with latent-context sharing; its outcome is the sum of two effects that, as we show, point in opposite directions. i i Figure 1: Overview of the proposed framework and study region. (i) End-to-end workflow of the study. Raw AIS trajectories are represented as a masked graph MDP; state features are selectively hidden to create partially observable settings; and reward functions learned under full and partial observability are compared using reward recovery, trajectory fidelity, and out-of-distribution generalization. (i) Study area with AIS trajectories discretized onto the H3 hexagonal grid. Figure 1(i) summarizes the methodology. We convert AIS voyages into demonstrations on a masked hexagonal graph MDP. We then test whether vessel differences remain after conditioning on observable factors. Finally, we compare reward models that isolate reward expressiveness (referred to as reward capacity) and latent context. We evaluate them using held-out prediction, route fidelity, and their ability to train a new policy. Trajectory Corpus and Preprocessing We use AIS data for cargo and tanker vessels operating in the Canadian Arctic during the July–October seasons of 2016–2024. We clean the positional streams and segment them into individual voyages, which serve as expert demonstrations. Following Spadon et al. (2025), we discretize each trajectory onto the H3111An open-source hierarchical hexagonal geospatial index; see https://h3geo.org/. hexagonal spatial index at resolution 6, where each cell is approximately 6.96.9 km wide. Figure 1(i) shows AIS voyages after discretization onto the H3 grid. We collapse repeated cells, bridge single-cell gaps, discard voyages with fewer than five transitions, and cap episode length at H=512H=512 steps. The resulting navigation graph contains ||=14,206|S|=14,206 water states and 39,05339,053 edges. The final dataset contains 3,1863,186 voyage demonstrations from 202202 vessels: 2,3752,375 cargo voyages and 811811 tanker voyages. Navigation MDP and State Representation We model each voyage as a goal-conditioned MDP (Liu et al. 2022). Each episode has a start cell and a goal cell, and the vessel moves between adjacent H3 cells. The action mask M(s,a)M(s,a) (Huang and Ontañón 2022) removes unavailable moves at coastal and pentagon cells. Our environment follows the graph-navigation setting of Vaidheeswaran et al. (2025). Their agent selects both direction and speed, whereas we model route direction only because speed does not change graph transitions. The action space has ||=7|A|=7 actions. Action 0 keeps the vessel in its current cell, and actions 11–66 move to neighboring cells in clockwise bearing order from north. At cells with fewer than six valid neighbors, M(s,a)M(s,a) masks unavailable actions by setting their logits to −109-10^9. We recover actions from consecutive cell pairs, so the demonstrations and trained models use the same MDP. We set γ=0.99γ=0.99. All models use ϕ(s)∈ℝ20φ(s) ^20: five environmental features per cell and month, including ERA5 wind and waves (Hersbach et al. 2020) and ORAS5 ice thickness, concentration, and temperature (Zuo et al. 2019); six geometric features for location and goal relation; and nine static vessel features from a 414414-vessel registry. We apply log(1+x) (1+x) to ice variables, standardize the remaining features using training-split statistics, and clip them at ±5σ± 5σ. Experimental Setup We use vessel-disjoint splits, ensuring that no vessel appears in more than one of the training, validation, and test sets. This evaluates generalization to unseen demonstrators. We additionally evaluate temporal robustness using a temporal-shift split that holds out the final months of trajectories from vessels observed during training. The resulting splits contain 1,939/246/833/1681,939/246/833/168 episodes and 134/20/41/39134/20/41/39 vessels for train, validation, test, and temporal-shift, respectively. All models are evaluated on the same held-out query decisions. For methods that infer latent context, three episodes per test vessel are reserved as support trajectories for estimating z and are excluded from evaluation. Reward Models and Baselines We use capacity-matched comparisons to separate nonlinear reward capacity from per-vessel latent context. MCE-IRL versus AIRL tests nonlinear capacity with a shared reward, whereas AIRL versus PEMIRL tests latent context at fixed nonlinear architecture, optimizer, and training budget. MCE-IRL learns a shared linear reward rθ(s)=θ⊤ϕ(s)r_θ(s)=θ φ(s) by maximizing demonstration likelihood under the soft-optimal policy in Eq. 1. AIRL learns a shared nonlinear reward adversarially against a PPO policy. PEMIRL augments AIRL with a recurrent inference network that infers a vessel-specific Gaussian context z∈ℝ8z ^8 from support trajectories and conditions both f(s,a,z)f(s,a,z) and π(a∣s,z)π(a s,z) on z (Chen et al. 2023). We additionally evaluate canonical AIRL (Fu et al. 2018), GAIL (Ho and Ermon 2016), BC, sequence-policy baselines (LSTM and Transformer), and the original PEMIRL implementation (Yu et al. 2019). All methods use the same data, splits, action masking, and evaluation protocol. Learned Rewards Evaluation We evaluate learned rewards along three complementary dimensions. Predictive accuracy is measured by the log-likelihood of held-out expert actions, reported with both decision-level and vessel-level aggregation. Behavioral fidelity measures whether the learned reward reproduces realistic navigation trajectories on held-out origin–destination pairs using trajectory similarity and task-completion metrics. Reward transfer measures whether the learned reward generalizes beyond the training policy by freezing the reward, training a masked PPO agent from scratch, and evaluating its navigation performance. All stochastic results are reported as mean ± standard deviation over three random seeds. Results Our central finding is that behavioral heterogeneity alone does not imply useful latent context. Once a shared reward has sufficient nonlinear capacity to model observable route and environmental factors, adding per-vessel context provides no benefit for held-out prediction, route generation, or reward transfer. Nonlinear Reward Capacity Versus Latent Context We first test whether PEMIRL’s predictive gains arise from its per-vessel latent context or from increased reward capacity. The standard comparison between PEMIRL and MCE-IRL changes both factors simultaneously, making the source of improvement ambiguous. We therefore introduce AIRL as a capacity-matched control. AIRL uses the same nonlinear reward architecture, optimizer, and training budget as PEMIRL, but removes the per-vessel latent context and its associated inference objective. Under the standard comparison, PEMIRL improves over MCE-IRL on unseen vessels (Table 1). However, the capacity-matched comparison reveals that the gain comes from nonlinear reward capacity rather than latent context. AIRL improves over MCE-IRL by 30.3%±4.1%30.3\%± 4.1\% per decision and 50.9%±1.1%50.9\%± 1.1\% per vessel without latent context, whereas adding PEMIRL’s latent context reduces likelihood relative to AIRL. Feature-expectation error follows the same ordering, with AIRL achieving the lowest error. These results show that PEMIRL’s apparent advantage over MCE-IRL is primarily explained by nonlinear reward capacity rather than by vessel-specific latent context. Per-vessel comparisons across the 4141 test vessels show the same ordering, with AIRL outperforming MCE-IRL on 3636 of 4141 vessels while PEMIRL does not consistently improve over AIRL. Temporal-shift, support-trajectory, and trajectory-length checks preserve the same conclusion. The main benefit of AIRL is reducing extreme failures: on matched test-split queries, the worst-performing MCE-IRL vessel scores −5.04-5.04 nats per decision, while AIRL improves the same vessel to −1.51-1.51, with no AIRL vessel below −2.28-2.28. AIRL improves over MCE-IRL on 36/4136/41 test vessels. We further compare against imitation models using the same held-out per-decision likelihood metric. Behavior cloning (BC) and sequence models achieve higher predictive likelihood, with the Transformer reaching −0.93-0.93 and the LSTM −0.82-0.82. This indicates that within-voyage history captures predictive information more effectively than PEMIRL’s cross-voyage latent context. GAIL reaches −1.38-1.38, below BC under the same protocol, consistent with the distinction between occupancy matching and direct next-action prediction (Orsini et al. 2021). Route Decoding and Reward Transfer Table 1: Comparison of reward and imitation models under a shared evaluation protocol. L denotes held-out per-decision log-likelihood. Results are mean ± std over seeds 0,1,2\0,1,2\. Transfer requires a frozen scalar training signal: BC and sequence policies provide only policies, whereas GAIL transfers its discriminator-derived surrogate −log(1−D(s,a))- (1-D(s,a)). Route decoding is restricted to rewards evaluated through the MCE-style soft-value-iteration decoder. Model Recovered reward L micro ↑ L macro ↑ Hausd. (km) ↓ Reach ↑ Transfer ↑ Masked-uniform chance — −1.90-1.90 — — — 0.09± 0.000.09 \,±\,0.00 MCE-IRL linear, shared −1.76± 0.01-1.76 \,±\,0.01 −1.85± 0.06-1.85 \,±\,0.06 45.6045.60 0.000.00 0.18± 0.010.18 \,±\,0.01 AIRL nonlinear, shared −1.49± 0.02-1.49 \,±\,0.02 −1.44± 0.06-1.44 \,±\,0.06 231.30± 6.20231.30 \,±\,6.20 0.16± 0.050.16 \,±\,0.05 0.59± 0.010.59 \,±\,0.01 PEMIRL nonlinear, per-vessel −1.62± 0.04-1.62 \,±\,0.04 −1.62± 0.06-1.62 \,±\,0.06 250.90± 4.80250.90 \,±\,4.80 0.04± 0.020.04 \,±\,0.02 0.59± 0.050.59 \,±\,0.05 Canonical AIRL shaped, state-only g −1.43± 0.01-1.43 \,±\,0.01 −1.36± 0.02-1.36 \,±\,0.02 233.30± 16.90233.30 \,±\,16.90 0.45± 0.020.45 \,±\,0.02 0.81± 0.060.81 \,±\,0.06 Original meta-IRL impl. nonlinear, per-vessel −1.29± 0.03-1.29 \,±\,0.03 −1.14± 0.03-1.14 \,±\,0.03 176.60± 15.40176.60 \,±\,15.40 0.53± 0.020.53 \,±\,0.02 0.35± 0.240.35 \,±\,0.24 GAIL discriminator surrogate (not IRL reward) −1.38± 0.01-1.38 \,±\,0.01 −1.30± 0.02-1.30 \,±\,0.02 — — 0.62± 0.110.62 \,±\,0.11 BC-MLP none (imitation) −1.26± 0.01-1.26 \,±\,0.01 −1.17± 0.02-1.17 \,±\,0.02 — — — seq-Transformer none (imitation) −0.93± 0.00-0.93 \,±\,0.00 −0.83± 0.01-0.83 \,±\,0.01 — — — seq-LSTM none (imitation) −0.82± 0.01-0.82 \,±\,0.01 −0.77± 0.03-0.77 \,±\,0.03 — — — Figure 2: Reward-derived evaluation criteria rank models differently across held-out action likelihood, decoded-route fidelity, and frozen-signal policy transfer. MCE-IRL produces the most realistic routes but performs worst on prediction and transfer, while nonlinear reward formulations reverse this trend. PEMIRL offers no improvement over context-free AIRL on any criterion. We next evaluate whether each recovered reward generates realistic full routes and can train a new policy from scratch. The model ranking changes substantially across these criteria (Table 1; Figure 2). Route decoding does not follow the likelihood ranking. MCE-IRL produces the most realistic complete routes, with 45.645.6 km Hausdorff distance and length ratio 0.770.77, whereas AIRL and PEMIRL produce trajectories 1212–1414 times the correct length (Figure 3). Their nearly identical route metrics indicate that this failure stems from the nonlinear reward formulation rather than latent context. The ordering also persists when both rewards are scored with MCE-IRL’s soft value iteration (947947 km for AIRL and 1,0301,030 km for PEMIRL). Thus, improved next-step prediction does not imply realistic long-horizon rollouts. Reward transfer gives a different result. A newly initialized PPO agent trained on a frozen nonlinear reward reaches held-out goals at steady state on 59.1%±1.1%59.1\%± 1.1\% of episodes for AIRL and 59.2%±5.2%59.2\%± 5.2\% for PEMIRL, compared with 8.9%8.9\% for a uniform-random policy. The corresponding jointly trained PEMIRL generator reaches only 4.2%±1.5%4.2\%± 1.5\%, showing that a recovered reward can support a new policy even when the policy trained alongside it does not. Figure 3 shows both effects on six held-out origin–destination pairs: MCE-IRL follows the correct corridor but never terminates at the goal, whereas agents trained on the frozen AIRL and PEMIRL rewards reach the goal along less direct routes and are visually indistinguishable from each other. GAIL provides an additional transfer control because its discriminator defines a frozen surrogate, −log(1−D(s,a))- (1-D(s,a)): PPO trained on this signal reaches held-out goals at 61.7%±10.6%61.7\%± 10.6\%. BC and sequence models provide only conditional policies, not a scalar reward to freeze for PPO; we omit GAIL route decoding because our decoder requires an MCE-style soft-value-iteration reward. MCE-IRL transfers poorly under the same optimization procedure. Its goal-reaching rate peaks at 0.3850.385 and falls to 0.1750.175 as PPO learns to loiter in high-reward regions rather than reach the goal. This is consistent with reward hacking under optimization pressure (Amodei et al. 2016; Skalse et al. 2022). We finally test whether this conclusion depends on implementation. Canonical AIRL (Eq. 2) predicts better than AIRL and PEMIRL (−1.434±0.012-1.434± 0.012) and transfers best (0.809±0.0610.809± 0.061 for its state-only reward g). Conversely, the original meta-IRL implementation gives the strongest likelihood (−1.288±0.026-1.288± 0.026) and highest decoded goal reach (0.530±0.0170.530± 0.017), but its frozen reward transfers unreliably (0.350±0.2400.350± 0.240) across seeds. Reliable reward transfer depends more on parameterization and optimization stability than on latent-context modeling. Figure 3: Decoded routes for six held-out origin–destination pairs. Each panel compares the real AIS trajectory with the MCE-IRL-decoded route and with fresh PPO agents trained on the recovered rewards from AIRL and PEMIRL. Hausdorff distance to the real trajectory is reported for each route. MCE-IRL is shown for corridor comparison despite not reaching the goal. Sources of Behavioral Heterogeneity We next ask why PEMIRL does not benefit from per-vessel context despite substantial vessel-level heterogeneity. Vessel identity explains η2=0.33η^2=0.33 of the variance in behavioral descriptors (Lakens 2013), far more than any category label (η2=0.085η^2=0.085 for size class), and η2=0.34η^2=0.34 after excluding short-haul voyages. However, this heterogeneity is largely explained by observable route and environmental factors already available to the shared reward. Across 123123 vessels with at least five voyages, only speed is strongly heterogeneous by vessel (F=2.47F=2.47); route-shape descriptors remain homogeneous (F≤0.38F≤ 0.38). Vessels therefore follow similar paths but move at different speeds. Grouping the same voyages by origin–destination route produces more variation than grouping them by vessel (F=2.45F=2.45 versus 0.830.83). This pattern mirrors the variable-centered versus person-centered distinction (Laursen and Hoff 2006; Howard and Hoffman 2018): PEMIRL’s per-vessel context is a person-centered representation, while AIRL’s shared nonlinear reward is variable-centered. Here, the person-centered grouping is the weaker one; the apparent vessel effect is primarily a route-assignment effect. On the route-matched subset (8282 routes with at least three voyages, n=330n=330), removing each route’s mean drops the vessel-level F-ratio from 1.031.03 to 0.230.23 overall and from 3.663.66 to 0.440.44 for speed, indicating that the apparent vessel effect primarily reflects route-assignment. Sea ice provides an observable explanation for this pattern. The same corridor is traversed quickly in open water and slowly under heavy ice. Consistent with this account, an ice-free Gulf of St. Lawrence fleet shows no heterogeneous grouping with F>1F>1. Route and ice are included in ϕ(s)φ(s), so a nonlinear shared reward can represent their effects without identifying the vessel. This explains why AIRL benefits from additional reward capacity while PEMIRL gains little from latent context. The null performance effect does not arise because the latent context failed to train. Across 4141 test vessels, the standard deviation of the posterior mean ranges from 0.0280.028 to 0.0750.075 over latent dimensions, and changing z alone changes the reward by 60%60\% as much as changing the full state–action input. A linear probe predicts the nine observable vessel-static features from z with R2=0.63R^2=0.63. The learned context is therefore active, but it primarily re-encodes information already available to the shared reward. We next test whether this redundancy is caused by vessel metadata in the observation. Removing all nine static vessel features leaves an 1111-dimensional ϕ(s)φ(s), creating a setting favorable to latent-context. Yet AIRL remains stronger than PEMIRL (−1.383-1.383 versus −1.566-1.566 macro likelihood), while AIRL itself does not degrade relative to the full-state setting (−1.441-1.441). With metadata hidden, vessel identity is weakly recoverable from behavior alone (R2=0.14R^2=0.14). Finally, the ice-free Gulf of St. Lawrence replication follows the same prediction. The capacity gain shrinks to +8.5%+8.5\%, compared with +30.3%+30.3\% in the Arctic, while the latent-context step remains negative (−12.8%-12.8\%). Across both regions, context fails when observable factors account for the relevant behavioral variation. These results suggest that demonstrator heterogeneity warrants latent context only when it remains unexplained after conditioning on observable state, route, and environment. A Practical Context-Need Diagnostic The preceding analyses suggest a diagnostic for deciding whether latent context is likely to add information beyond the observed state. This diagnostic first measures demonstrator-level heterogeneity, identifies the descriptors in which it occurs, and recomputes it after conditioning on observed covariates. In our setting, this isolates speed as the primary heterogeneous descriptor (F=2.47F=2.47 versus F≤0.38F≤ 0.38 for route shape) and shows that conditioning on route substantially reduces this variation (F:1.03→0.23F:1.03→ 0.23 overall, 3.66→0.443.66→ 0.44 for speed). Persistent residual variation would motivate a latent-context model, whereas variation that largely disappears favors a capacity-matched shared reward as the primary baseline. When a latent-context model is trained, the learned context should also be audited rather than treated as evidence of useful hidden structure by itself. Posterior variation and reward sensitivity establish whether the context influences the model, while decoding observed covariates tests whether it captures additional information or re-encodes the observed state. In our experiments, the learned context influences the reward yet largely reflects observable vessel properties, and the hidden-covariate ablation confirms that this decoding is not solely due to information already exposed in the state. Together, these analyses distinguish genuinely hidden behavioral structure from latent representations that primarily capture observable features. Discussion The central distinction is between heterogeneity explained by the observed state and heterogeneity that remains unobserved. Demonstrators may differ substantially while those differences are attributable to route, environment, or other variables already available to a shared reward. In such settings, a latent-context formulation need not improve reward learning, even when its inferred context is active. Latent context is most compelling when it captures decision-relevant variation that the state does not adequately represent. This is a property of both the dataset and the state representation. The same trajectories could support a different modeling choice if relevant environmental variables were omitted or if unobserved factors, such as operator intent, influenced behavior. Similar considerations arise in other real-world settings, such as autonomous driving, where observed differences may reflect road and traffic conditions rather than unobserved driver preferences. Implications for Evaluation. Latent-context IRL should be evaluated against a capacity-matched shared control. Comparing a context-conditioned model only with a simpler shared baseline can confound the value of nonlinear reward capacity with the value of latent context. Evaluation should also extend beyond held-out next-action likelihood. Predictive accuracy, long-horizon route decoding, and reward transfer measure different properties of a learned system and may favor different models. Evaluating a frozen reward under a newly trained policy provides a direct test of how it behaves under downstream optimization. Scope and Future Directions. Our conclusions concern per-vessel context in a direction-only route-choice MDP. Speed accounts for much of the observed vessel-level variation but is not an action in this formulation, so the results do not establish whether latent information could improve operational speed control or joint route–speed planning. Other task definitions may also be more appropriate when variation is indexed by route, cargo, company, season, or time rather than vessel identity. Likewise, partially observed factors such as operator intent may require history-conditioned policies or belief states instead of a static per-vessel context. Hierarchical reward models, constraint-based formulations, and models that explicitly represent dynamics are complementary ways to model temporal subgoals, operational constraints, or environment mismatch. Implications for Arctic Navigation. For AI-assisted Arctic route analysis and decision support, our results favor adaptation through observed, decision-relevant conditions over unvalidated per-vessel latent representations. Known operational requirements should be encoded as explicit constraints, enabling transparent and auditable recommendations. Per-vessel latent context should be introduced only when diagnostic evidence shows that observable features cannot explain behavioral variation. This provides a principled basis for developing accountable AI decision-support systems in safety-critical Arctic operations. Limitations. This study uses cargo and tanker voyages in a single graph MDP, so the findings may not generalize to other environments, feature representations, or reward formulations. Reward-transfer results are specific to the MDP and PPO configuration. Finally, observed trajectories reflect behavior rather than underlying operational intent, so cross-domain evaluation is needed to determine when latent context provides benefits beyond observable state information. Conclusion This work shows that observed behavioral differences alone do not justify the introduction of latent-context inverse reinforcement learning. Through a controlled evaluation in Arctic shipping, we separate the benefits of expressive reward modeling from those of per-vessel latent context and show how an observability analysis can help assess whether personalization is empirically warranted. For AI-assisted navigation, this highlights the importance of grounding adaptive systems in the information available from the operating environment before attributing behavior to hidden factors. Latent representations should be introduced only when they provide additional decision-relevant information beyond observable conditions. More broadly, this work supports the evidence-based use of latent-context models, in which additional model complexity should be justified by information not already represented in the observed state. Acknowledgments This work was supported by the Natural Sciences and Engineering Research Council of Canada — NSERC, the Faculty of Computer Science of Dalhousie University, and the Conselho Nacional de Desenvolvimento Científico e Tecnológico — CNPq, Brasil. Responsible AI Use Generative AI technologies supported initial manuscript drafting and the creation of preliminary code scaffolding. The authors independently reviewed, verified, and revised all AI-assisted material and retain full responsibility for the study’s methods, claims, analyses, interpretations, figures, and reported results. References D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané (2016) Concrete problems in AI safety. Note: arXiv:1606.06565 External Links: Document Cited by: Related Work, Route Decoding and Reward Transfer. J. Chen, D. Tamboli, T. Lan, and V. Aggarwal (2023) Multi-task hierarchical adversarial inverse reinforcement learning. In Proc. ICML, PMLR, Vol. 202, p. 4895–4920. Cited by: Reward Models and Baselines. X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel (2016) InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In Adv. Neural Inf. Process. Syst., Vol. 29, p. 2172–2180. Cited by: Related Work, Preliminaries. C. Finn, P. Abbeel, and S. Levine (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In Proc. ICML, PMLR, Vol. 70, p. 1126–1135. Cited by: Related Work. C. Finn, S. Levine, and P. Abbeel (2016) Guided cost learning: deep inverse optimal control via policy optimization. In Proc. ICML, PMLR, Vol. 48, p. 49–58. Cited by: Related Work. J. Fu, K. Luo, and S. Levine (2018) Learning robust rewards with adversarial inverse reinforcement learning. In Proc. ICLR, Cited by: Introduction, Related Work, Preliminaries, Reward Models and Baselines. J. J. Heckman (1981) Heterogeneity and state dependence. In Studies in Labor Markets, S. Rosen (Ed.), p. 91–140. External Links: Document Cited by: Related Work. H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Horányi, J. Muñoz-Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers, A. Simmons, C. Soci, S. Abdalla, X. Abellan, G. Balsamo, P. Bechtold, G. Biavati, J.-R. Bidlot, M. Bonavita, G. De Chiara, P. Dahlgren, D. Dee, M. Diamantakis, R. Dragani, J. Flemming, R. Forbes, M. Fuentes, A. Geer, L. Haimberger, S. Healy, R. J. Hogan, E. Hólm, M. Janisková, S. Keeley, P. Laloyaux, P. Lopez, C. Lupu, G. Radnoti, P. de Rosnay, I. Rozum, F. Vamborg, S. Villaume, and J.-N. Thépaut (2020) The ERA5 global reanalysis. Q. J. R. Meteorol. Soc. 146 (730), p. 1999–2049. External Links: Document Cited by: Navigation MDP and State Representation. J. Ho and S. Ermon (2016) Generative adversarial imitation learning. In Adv. Neural Inf. Process. Syst., Vol. 29, p. 4565–4573. Cited by: Related Work, Reward Models and Baselines. M. C. Howard and M. E. Hoffman (2018) Variable-centered, person-centered, and person-specific approaches: where theory meets the method. Organ. Res. Methods 21 (4), p. 846–876. External Links: Document Cited by: Sources of Behavioral Heterogeneity. S. Huang and S. Ontañón (2022) A closer look at invalid action masking in policy gradient algorithms. In Proc. FLAIRS, Vol. 35. External Links: Document Cited by: Navigation MDP and State Representation. D. Lakens (2013) Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front. Psychol. 4, p. 863. External Links: Document Cited by: Sources of Behavioral Heterogeneity. B. Laursen and E. Hoff (2006) Person-centered and variable-centered approaches to longitudinal data. Merrill-Palmer Q. 52 (3), p. 377–389. External Links: Document Cited by: Sources of Behavioral Heterogeneity. Y. Li, J. Song, and S. Ermon (2017) InfoGAIL: interpretable imitation learning from visual demonstrations. In Adv. Neural Inf. Process. Syst., Vol. 30, p. 3812–3822. Cited by: Introduction, Related Work. M. Liu, M. Zhu, and W. Zhang (2022) Goal-conditioned reinforcement learning: problems and solutions. In Proc. IJCAI, p. 5502–5511. External Links: Document Cited by: Navigation MDP and State Representation. F. L. Mannering, V. Shankar, and C. R. Bhat (2016) Unobserved heterogeneity and the statistical analysis of highway accident data. Anal. Methods Accid. Res. 11, p. 1–16. External Links: Document Cited by: Related Work. D. Nguyen and R. Fablet (2024) A transformer network with sparse augmented data representation and cross entropy loss for AIS-based vessel trajectory prediction. IEEE Access 12, p. 21596–21609. External Links: Document Cited by: Related Work. M. Orsini, A. Raichuk, L. Hussenot, D. Vincent, R. Dadashi, S. Girgin, M. Geist, O. Bachem, O. Pietquin, and M. Andrychowicz (2021) What matters for adversarial imitation learning?. In Adv. Neural Inf. Process. Syst., Vol. 34, p. 14656–14668. Cited by: Related Work, Nonlinear Reward Capacity Versus Latent Context. T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters (2018) An algorithmic perspective on imitation learning. Found. Trends Robot. 7 (1–2), p. 1–179. External Links: Document Cited by: Related Work. K. Rakelly, A. Zhou, C. Finn, S. Levine, and D. Quillen (2019) Efficient off-policy meta-reinforcement learning via probabilistic context variables. In Proc. ICML, PMLR, Vol. 97, p. 5331–5340. Cited by: Related Work. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. Note: arXiv:1707.06347 External Links: Document Cited by: Preliminaries. Y. Shu, W. Xu, H. Cui, J. Xiao, L. Song, H. Li, and Z. Yang (2026) Integrating spatio-temporal analysis for assessing the effectiveness of POLARIS in arctic shipping traffic. Transp. Policy 181, p. 104096. External Links: Document Cited by: Introduction, Related Work. J. Skalse, N. Howe, D. Krasheninnikov, and D. Krueger (2022) Defining and characterizing reward gaming. In Adv. Neural Inf. Process. Syst., Vol. 35, p. 9460–9471. External Links: Document Cited by: Related Work, Route Decoding and Reward Transfer. G. Spadon, R. Song, V. Vaidheeswaran, M. M. Alam, F. Goerlandt, and R. Pelot (2025) Modeling maritime transportation behavior using AIS trajectories and Markovian processes in the gulf of st. lawrence. In Proc. IEEE Big Data, p. 5314–5323. External Links: Document Cited by: Related Work, Trajectory Corpus and Preprocessing. V. Vaidheeswaran, D. Jayakody, S. Mulay, A. Lo, M. M. Alam, and G. Spadon (2025) Goal-conditioned reinforcement learning for data-driven maritime navigation. In Proc. IEEE Big Data, p. 1194–1203. External Links: Document Cited by: Related Work, Navigation MDP and State Representation. J. Wang, N. Wu, W. X. Zhao, F. Peng, and X. Lin (2019) Empowering A* search algorithms with neural networks for personalized route recommendation. In Proc. ACM SIGKDD, p. 539–547. External Links: Document Cited by: Related Work. K. Xu, E. Ratner, A. Dragan, S. Levine, and C. Finn (2019) Learning a prior over intent via meta-inverse reinforcement learning. In Proc. ICML, PMLR, Vol. 97, p. 6952–6962. Cited by: Introduction, Related Work. L. Yu, T. Yu, C. Finn, and S. Ermon (2019) Meta-inverse reinforcement learning with probabilistic context variables. In Adv. Neural Inf. Process. Syst., Vol. 32, p. 11772–11783. Cited by: Introduction, Introduction, Related Work, Preliminaries, Reward Models and Baselines. C. Zhang, D. Zhang, M. Zhang, J. Zhang, and W. Mao (2022) A three-dimensional ant colony algorithm for multi-objective ice routing of a ship in the arctic area. Ocean Eng. 266, p. 113241. External Links: Document Cited by: Related Work. Y. Zheng (2015) Trajectory data mining: an overview. ACM Trans. Intell. Syst. Technol. 6 (3), p. 29:1–29:41. External Links: Document Cited by: Related Work. B. D. Ziebart, J. A. Bagnell, and A. K. Dey (2010) Modeling interaction via the principle of maximum causal entropy. In Proc. ICML, p. 1255–1262. Cited by: Introduction, Introduction, Related Work, Preliminaries. B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey (2008) Maximum entropy inverse reinforcement learning. In Proc. AAAI, p. 1433–1438. Cited by: Introduction, Related Work. H. Zuo, M. A. Balmaseda, S. Tietsche, K. Mogensen, and M. Mayer (2019) The ECMWF operational ensemble reanalysis-analysis system for ocean and sea ice: a description of the system and assessment. Ocean Sci. 15 (3), p. 779–808. External Links: Document Cited by: Navigation MDP and State Representation. Appendix Appendix A A. Reproducibility Details for Data and Evaluation The main paper specifies the trajectory corpus, masked H3 graph MDP, state representation, and vessel-disjoint split design. This section records implementation details and sample counts needed to reproduce the reported evaluations. Graph and action masking. The Arctic navigation graph spans 60.0∘60.0 –77.0∘77.0 N and 127.9∘127.9 –64.5∘64.5 W and contains 14,20614,206 navigable H3 resolution-6 cells and 39,05339,053 undirected adjacency edges. The action space has seven actions: action 0 remains in the current cell, and actions 11–66 select neighboring cells in clockwise bearing order from north. At coastal and pentagon-boundary cells, unavailable actions are masked by adding −109-10^9 to their logits. Thus, masked actions have zero probability in every policy, likelihood calculation, chance baseline, and decoded route. We recover actions from consecutive discretized cells and use H=512H=512 and γ=0.99γ=0.99 throughout. Preprocessing and features. The corpus comprises cargo and tanker AIS trajectories from July–October of 2016–2024. After voyage segmentation, H3 discretization, removal of repeated cells, bridging of single-cell gaps, and exclusion of voyages with fewer than five transitions, it contains 3,1863,186 demonstrations (2,3752,375 cargo; 811811 tanker). Each state representation has 20 components: five monthly ERA5/ORAS5 environmental variables, six location- and goal-relative geometric terms, and nine vessel-registry attributes. Sea-ice variables use log(1+x) (1+x); all other features are standardized using training-split statistics and clipped to ±5σ± 5σ. Missing registry attributes, affecting approximately 7%7\% of demonstration vessels, are set to zero. Evaluation populations. Training, validation, and test splits are vessel-disjoint. The temporal shift uses later months from vessels represented in training. For context-conditioned models, three episodes per evaluation vessel are reserved as support trajectories and are never included in likelihood scoring. All model comparisons use the matched query population: only decisions scored by every model under the relevant protocol are retained. Table 2: Dataset splits and matched query populations. The train, validation, and test splits are vessel-disjoint and together cover 3,0183,018 voyages from 195195 vessels; the temporal-shift split holds the remaining 168168 voyages of the 3,1863,186-voyage corpus. Split Episodes Vessels Decisions Query dec. Train 1,9391,939 134134 151,782151,782 — Validation 246246 2020 14,89414,894 — Test 833833 4141 70,99570,995 62,12762,127 Temporal shift 168168 39†39 17,41117,411 8,5088,508 †32 are training vessels evaluated in unseen months; the other 7 occur in no other split, giving 202 corpus-wide. Appendix B B. Training and Optimization Details This section gives the settings needed to reproduce model training. All models use the same MDP, action masks, state features, vessel-disjoint splits, matched query sets, and random seeds 0,1,2\0,1,2\. AIRL and PEMIRL are trained on the same 1,000 demonstrations. Table 3 lists the main settings. MCE-IRL training. We train MCE-IRL with Adam for 200 iterations. We select the model checkpoint with the best validation log-likelihood. This checkpoint occurs at iteration 40 and has validation log-likelihood −1.754-1.754. The learned parameter vectors are highly consistent across seeds, with pairwise cosine similarity above 0.99990.9999. For each demonstration group, soft value iteration uses the corresponding goal, year, month, vessel, horizon, and environmental features. Thus, the policy is evaluated under the conditions of the voyage being modeled. AIRL and PEMIRL training. AIRL and PEMIRL are trained for 500 outer iterations without early stopping. Each iteration generates 64 rollouts. PEMIRL samples 16 latent contexts and generates four rollouts for each context. For reward-transfer experiments, we freeze the learned reward and train a new PPO agent from scratch. PPO uses 200 iterations with 64 rollouts per iteration. We compute all reported evaluation metrics on CPU. PEMIRL stability settings. The original PEMIRL information-maximization objective was unstable in our graph-navigation setting. In particular, the objective uses the sum of rewards along a trajectory, and this sum can grow without bound. We therefore use the following stability settings in all default PEMIRL experiments: 1. Clamp the trajectory return used by the information objective to the range [−10,10][-10,10]. 2. Standardize the information-objective advantage within each minibatch. 3. Clip discriminator and posterior-network gradient norms to 1.01.0. 4. Use the discriminator logit f directly as the PPO reward, rather than using σ(f)σ(f). 5. Set the information-maximization weight to η=0.01η=0.01. 6. Start posterior-network updates after the first 10 outer iterations. For the information-maximization sensitivity analysis, we change only the weight to η=0.1η=0.1 and retain the other five settings. Table 3: Main training and architecture settings. GAE denotes generalized advantage estimation. The latent clamp bounds sampled PEMIRL context values. Component Setting Value MCE-IRL Learning rate 0.050.05 ℓ2 _2 penalty 10−410^-4 Training iterations 200200 AIRL Discriminator arch. MLP (256,256)(256,256) Generator arch. MLP (256,256)(256,256) Gradient-penalty weight 1010 PEMIRL Latent dimension 88 Latent clamp ±2± 2 Posterior arch. Bi-LSTM (128)(128) Information weight η 0.010.01 Return clamp 1010 Posterior warm-up 1010 iterations Adversarial Outer iterations 500500 Rollouts per iteration 6464 Training demonstrations 1,0001,000 Random seeds 0,1,2\0,1,2\ PPO Clip ratio 0.20.2 GAE λ 0.950.95 Optimization epochs 88 Entropy coefficient 0.010.01 Value coefficient 0.50.5 Learning rate 3×10−43× 10^-4 Reward transfer Support episodes per vessel 33 PPO iterations 200200 Rollouts per iteration 6464 Appendix C C. Sensitivity Analysis of the Main Comparison This section tests whether the main comparison changes under different evaluation splits, aggregation methods, support-trajectory choices, or trajectory-length filtering. The main paper reports the central comparison; here we provide the full results. Evaluation splits and aggregation. Table 4 reports held-out action log-likelihood on the standard test split and the temporal-shift split. We report two averages. Micro aggregation gives equal weight to every decision. Macro aggregation gives equal weight to every vessel. We also report test feature-expectation error (FEE), where lower values are better. The same model ordering appears in each split and aggregation: AIRL has the best log-likelihood, followed by PEMIRL and MCE-IRL. The temporal-shift results therefore do not change the main conclusion. Support and horizon checks. Table 5 tests two possible sources of evaluation bias. First, it compares randomly selected support voyages with causal support voyages, defined as each vessel’s earliest available voyages. Second, it excludes 37 episodes that reach the horizon limit of 512 steps. These checks leave the model ordering unchanged. In particular, causal support does not improve PEMIRL enough to close its gap with AIRL. This suggests that the central result is not explained by information from later support voyages or by horizon-truncated episodes. Per-vessel performance. We also evaluate each test vessel separately. On the matched test set, the lowest MCE-IRL score is −5.04-5.04 nats per decision. AIRL obtains −1.51-1.51 nats on that same vessel, and its lowest score across all test vessels is −2.28-2.28 nats per decision. Complete per-vessel and per-episode results are available in runs/eval/per_episode_l*.json. Table 4: Held-out per-decision log-likelihood by split and aggregation (higher is better), reported as mean ± standard deviation across three seeds. The empirical masked-action chance log-likelihood is −1.90-1.90 on the test split and −1.91-1.91 on the temporal-shift split. FEE denotes test feature-expectation error (lower is better). Split Average MCE-IRL AIRL PEMIRL linear nonlinear ++ context Test micro −1.76± 0.01-1.76 \,±\,0.01 −1.49± 0.02-1.49 \,±\,0.02 −1.62± 0.04-1.62 \,±\,0.04 macro −1.85± 0.06-1.85 \,±\,0.06 −1.44± 0.06-1.44 \,±\,0.06 −1.62± 0.06-1.62 \,±\,0.06 Temporal shift micro −1.72± 0.04-1.72 \,±\,0.04 −1.47± 0.03-1.47 \,±\,0.03 −1.59± 0.04-1.59 \,±\,0.04 macro −1.85± 0.08-1.85 \,±\,0.08 −1.42± 0.05-1.42 \,±\,0.05 −1.60± 0.06-1.60 \,±\,0.06 Test FEE ↓ 826826 ± 35617 \,±\,35 677± 27677 \,±\,27 Table 5: Test log-likelihood under support and trajectory-length variants, averaged over three seeds. R uses randomly selected support voyages; C uses causal support, namely the earliest available voyages. −-T excludes the 37 horizon-truncated test episodes. The largest standard deviation across seeds is 0.0620.062. Average Model R R−-T C C−-T Macro MCE-IRL −1.852-1.852 −1.825-1.825 −1.849-1.849 −1.834-1.834 AIRL −1.441-1.441 −1.433-1.433 −1.461-1.461 −1.457-1.457 PEMIRL −1.621-1.621 −1.619-1.619 −1.590-1.590 −1.588-1.588 Micro MCE-IRL −1.757-1.757 −1.675-1.675 −1.752-1.752 −1.674-1.674 AIRL −1.492-1.492 −1.475-1.475 −1.496-1.496 −1.482-1.482 PEMIRL −1.620-1.620 −1.614-1.614 −1.620-1.620 −1.614-1.614 Appendix D D. Feature-Hiding Ablation This ablation tests whether latent context becomes useful when static vessel information is removed from the observed state. The full state contains 20 features, including nine vessel attributes. In the hidden setting, we remove all nine vessel attributes and retain 11 features: five environmental variables, normalized latitude, (cos,sin)( , ) longitude, distance to the goal, and (cos,sin)( , ) bearing to the goal. We adjust network input sizes to match the reduced feature vector. The context-conditioned discriminator receives 26 inputs, and the context-free discriminator receives 18 inputs. All other settings are unchanged: the dataset splits, feature standardization procedure, training budget, and random seeds are the same as in the full-feature experiments. Before running this experiment, we specified the hypothesis that latent context might compensate for omitted vessel metadata. This is the only preregistered analysis in the study. The heterogeneity analysis in Section F was developed after the main experiments. Results. Table 6 compares held-out test log-likelihood using the full and hidden state representations. Removing vessel attributes does not improve PEMIRL relative to AIRL. AIRL remains the best-performing model under both micro and macro aggregation. Table 6: Feature-hiding results on the test split. Values are mean per-decision log-likelihood across three seeds; higher is better. The largest seed standard deviation is 0.0630.063. Full (D=20D=20) Vessel feat. hidden (D=11D=11) Model Micro Macro Micro Macro MCE-IRL −1.757-1.757 −1.852-1.852 −1.896-1.896 −2.083-2.083 AIRL −1.492-1.492 −1.441-1.441 −1.447-1.447 −1.383-1.383 PEMIRL −1.620-1.620 −1.621-1.621 −1.573-1.573 −1.566-1.566 Why MCE-IRL cannot use these features. The removed vessel attributes are constant within a voyage. Therefore, a linear MCE-IRL reward can assign them only a constant reward offset within that voyage. This offset changes values but not action probabilities. Proposition. Let ϕ(s)=[ψ(s);m]φ(s)=[ψ(s);m], where m is constant for every state in an episode, and let rθ(s)=θ⊤ϕ(s)r_θ(s)=θ φ(s). The soft value iteration policy for that episode is independent of m and its corresponding parameter vector θm _m. Proof. Write rθ(s)=θψ⊤ψ(s)+cr_θ(s)= _ψ ψ(s)+c, where c=θm⊤mc= _m m is constant within the episode. At time t, adding c to every state reward adds the same quantity, c(H−t)c(H-t), to all action-values Qt(s,a)Q_t(s,a) and to the state value Vt(s)V_t(s). The policy depends on their difference: logπt(a∣s)=Qt(s,a)−Vt(s). _t(a s)=Q_t(s,a)-V_t(s). The shared offset therefore cancels. The same argument applies with the goal boundary condition Vt(g)=0V_t(g)=0. Hence, episode-constant metadata cannot change the MCE-IRL action policy. □ The MCE-IRL result should therefore not be interpreted as evidence that the removed vessel attributes directly supported route choice in the linear model. More importantly, hiding those attributes does not give PEMIRL an advantage over the capacity-matched AIRL baseline. Appendix E E. Context Audit Context probes are evaluated on the seed-0 PEMIRL checkpoint across the 41 test vessels using posterior mean z¯v z_v inferred from three support episodes (Table 7). Posterior spread measures the standard deviation of z¯v z_v across vessels. Reward sensitivity computes ratio σz/σsa _z/ _sa of reward standard deviation when varying z versus varying state-action pairs (s,a)(s,a). Decoding probes fit linear regression models from z¯v z_v to the nine static vessel features, reporting variance-weighted mean R2R^2 values under full and metadata-hidden feature sets. The measured posterior variation and feature decodability confirm that the context pathway is active and encodes observable vessel attributes. However, these metrics do not establish that the learned context provides useful hidden decision information for policy modeling. Table 7: Context probes evaluated on test vessels. σz _z measures reward variation over posterior means z¯v z_v with (s,a)(s,a) fixed; σsa _sa measures reward variation over state-action pairs with z fixed. Decoding probes report variance-weighted mean R2R^2 from linear regression of vessel statics on z¯v z_v. Probe Quantity Value Posterior spread std of z¯v z_v across vessels, per dim. 0.0280.028–0.0750.075 overall 0.0660.066 Reward sensitivity σz/σsa _z/ _sa 0.5990.599 Decoding, full state mean R2R^2, statics from z¯v z_v 0.630.63 Decoding, metadata hidden mean R2R^2 (per dim. 0.110.11–0.250.25) 0.140.14 Appendix F F. Heterogeneity Methods Per-voyage descriptors are calculated as follows: • Straightness: Great-circle origin to destination displacement divided by trajectory path length. • Turn rate: Mean absolute heading change per step. • Mean speed and speed standard deviation: Derived from consecutive cell timestamps. • Loiter fraction: Share of trajectory steps with speed below 0.50.5 knots. Effect sizes (η2η^2) are computed per grouping factor over 3,0183,018 voyages from 195195 vessels (Table 8). Bias-adjusted ε2 ^2 estimates (Olejnik and Algina 2003) preserve identical rank orderings. Variance ratios F=Varbetween/VarwithinF=Var_between/Var_within are calculated on z-scored descriptors. Inclusion thresholds require at least 5 voyages per group for vessel and route analyses, and at least 3 voyages for route-controlled analysis (8282 routes, n=330n=330 voyages). The route-controlled protocol subtracts each route’s mean descriptor value prior to grouping by vessel (Table 9). Table 8: Variance in per-voyage descriptors explained by grouping factors (η2η^2), computed over 3,0183,018 voyages from 195195 vessels. Grouping factor All voyages Short-haul excl. Vessel identity 0.3300.330 0.3410.341 Size class 0.0850.085 0.0840.084 Subtype 0.0190.019 0.0200.020 Cargo vs. tanker 0.0070.007 0.0060.006 Table 9: Decomposition of vessel variance. Left: per-descriptor F-ratios by vessel. Right: variance ratios for route versus vessel groupings, including route-controlled vessel effects (F>1F>1 indicates that between-group variance exceeds within-group variance). Descriptor F by vessel Grouping F straightness 0.140.14 by vessel 0.830.83 turn rate 0.380.38 by O–D route 2.452.45 mean speed 2.472.47 vessel, route-ctrl.: speed std 0.970.97 all descriptors 1.03→0.231.03 0.23 loiter fraction 0.170.17 mean speed 3.66→0.443.66 0.44 The secondary evaluation region is the Gulf of St. Lawrence (44.9∘44.9 –52.3∘52.3 N, 71.7∘71.7 –54.7∘54.7 W), discretized at H3 resolution 6 across all twelve months of 2024. Heterogeneity analysis covers 24,57224,572 voyages across nine vessel types. Model replication uses a goal-conditioned subset (3,2233,223 voyages, 603603 vessels, 7,6907,690 states, 20,48420,484 edges) under identical split logic. Because environmental reanalysis data are unavailable for this region, feature vectors ϕ(s)φ(s) include geometry, goal-relative terms, and vessel attributes only. Absolute likelihoods are not directly comparable to Arctic values, and only within-region model orderings are evaluated. Table 10 reports the replication. The within-region ordering matches the Arctic: AIRL is best, MCE-IRL second, and PEMIRL last. We express each step as the relative change in per-decision likelihood, exp(ΔL)−1 ( \,L)-1. The capacity step from MCE-IRL to AIRL is +8.5%+8.5\% on test and +3.7%+3.7\% under temporal shift, compared with +30.3%+30.3\% in the Arctic. The latent-context step from AIRL to PEMIRL is negative in both splits, −12.8%-12.8\% and −12.5%-12.5\%. The reduced capacity gain in an ice-free region, alongside an unchanged negative context step, is the pattern predicted by the heterogeneity analysis. These runs use a single seed and a reduced feature set, so they support the ordering claim only, not the Arctic effect magnitudes. Table 10: Gulf of St. Lawrence replication (single seed, reduced feature set). L is held-out per-decision log-likelihood, higher is better; FEE is feature-expectation error, lower is better. Absolute values are not comparable to the Arctic results in Table 4. Model Test L Temp. shift L Test FEE ↓ MCE-IRL −1.366-1.366 −1.292-1.292 863863 AIRL −1.285-1.285 −1.256-1.256 490 PEMIRL −1.422-1.422 −1.390-1.390 534534 Appendix G G. Reward Transfer Protocol The learned reward function is frozen, and a newly initialized action-masked PPO agent is trained for 200200 iterations of 6464 rollouts without access to expert demonstrations. For context-conditioned rewards, latent vector z is inferred once per vessel from three support episodes and held constant. Performance is measured as the fraction of held-out test episodes reaching the goal cell within H=512H=512 steps. Steady-state performance is defined as the mean goal-reaching rate over the final 20%20\% of training iterations (40 iterations). A uniform-random policy over valid actions serves as baseline, achieving a 0.0890.089 goal-reaching rate. Figure 4 illustrates transfer training dynamics. The linear reward exhibits non-monotonic transfer performance: goal-reaching efficiency peaks at 0.3850.385 before declining to a steady-state value of 0.1750.175. This drop occurs because the agent learns that remaining in high-reward states yields higher return than terminating at the goal. Nonlinear reward functions maintain stable goal-reaching performance throughout training. Figure 4: Reward transfer training curves, mean ± standard deviation over three seeds. Baseline reference lines show the uniform-random valid policy (0.0890.089) and the generator policy trained jointly with the PEMIRL reward (0.0250.025, seed 0). A potential-shaping diagnostic, in which the transferred signal is r(s,a,s′)+γV(s′)−V(s)r(s,a,s )+γ V(s )-V(s) rather than the raw reward, is reported for the original implementation in Section I. Appendix H H. Reproducibility and Artifact Map The code and data supplement contains no raw or processed AIS positional data, tracks, kinematics, or vessel names. Vessel identifiers are replaced with consistent opaque pseudonyms across all files. Artifacts marked with † in Table 11 are included in the supplement and can be inspected or re-derived without any AIS data. The remaining rows depend on the third-party licensed AIS corpus or on auxiliary experiment code. For those, the supplement ships the pipeline scripts and configuration files that document the exact procedure; the auxiliary experiment code is part of the full repository release. Table 11: Mapping of experimental results to repository artifacts. Artifacts marked with † are included in the code and data supplement and can be evaluated directly. Result Artifact L micro/macro (Tab. 4) † runs/eval/ per_episode_ l*.json, robustness* .json Per-vessel dispersion † runs/eval/ per_episode_ l.json, per_episode_ l_noctx.json Paired per-vessel tests † runs/eval/ significance.json Support variants (Tab. 5) † runs/eval/ support_sweep*.json Route fidelity † results/reference/ routes_*.csv Reward transfer (Fig. 4) † models/ ppo_on_pemirl/ history.json (seed 0); per-seed histories in full repository Effect sizes (Tab. 8) † results/reference/ eta_squared.json Feature hiding (Tab. 6) † runs/eval/ *_hidden*.json Info-max variants † runs/eval/ robustness_ info10.json Landscape comparison † runs/eval/ *canonical_airl*, *gail*, *bc*, *seq_*.json Context probes (Tab. 7) † models/pemirl/ pemirl.pt; probe script in full repository Stabilization recipe † configs/pemirl.yaml, with inline rationale F-ratios (Tab. 9) descriptor script 02_ heterogeneity.py; per-voyage descriptors withheld as AIS derivatives Second region regional configs configs/ gsl*.yaml and dataset builder Semi-synthetic control (Tab. 13) generation, training, and probe scripts (full repository) Original implementation adapter and export scripts (full repository) The context-conditioned discriminator, bi-LSTM posterior encoder, info-max objective, and training loop are adapted from the PEMIRL implementation of Chen et al. (2023) (model/pemirl_airl.py and model/context_net.py), modified for discrete, action-masked, goal-conditioned graph MDPs. Core loss functions and gradient penalty conventions are preserved, with stabilization modifications described in Section B. The replication analysis in Section I uses the original PEMIRL code release from Yu et al. (2019). Implementations of MCE-IRL (Ziebart et al. 2010), AIRL (Fu et al. 2018), GAIL (Ho and Ermon 2016), PPO (Schulman et al. 2017), and baseline models were developed from published specifications. Appendix I I. Faithful Replication of the Original Implementation The original PEMIRL codebase (Yu et al. 2019) uses TensorFlow 1 and targets continuous control with fixed-length trajectories. Replicating this implementation required four interface adaptations: 1. Demonstration formatting: Training voyages (1,0001,000) were converted to fixed-length trajectories of 100100 steps via truncation (26.8%26.8\% of episodes) or goal padding. 2. Environment adapter: A graph replay environment matching state transitions, action masks, and features was constructed under the original environment interface. 3. Masked discrete policy: The categorical policy was modified with action masking, offsetting invalid logits rather than setting them to −∞-∞ due to entropy calculations. 4. Fidelity gating: Stabilization modifications were placed behind configuration flags and disabled to evaluate the published objective. Data exchanged between TensorFlow 1 and PyTorch frameworks were transferred via numeric arrays. Trained networks were re-implemented in NumPy and verified against TensorFlow outputs to numerical floating-point precision. State sequences were verified by replaying all demonstrations through the adapter. The original configuration uses a 33-dimensional latent context, against 88 dimensions in our implementation. Results. Table 12 reports the three seeds. The original implementation obtains the best held-out likelihood of any model we evaluate, and the best goal-reaching rate for decoded routes, but it does not yield a transferable reward: a fresh PPO agent on its frozen reward reaches 0.0930.093 of held-out goals on seed 0, below the 0.0890.089 uniform-random baseline, and is highly seed-dependent (0.35±0.240.35± 0.24 across seeds). Transferring the potential-shaped signal r(s,a,s′)+γV(s′)−V(s)r(s,a,s )+γ V(s )-V(s) instead of the raw reward raises this to 0.43±0.210.43± 0.21 (seed 0: 0.1560.156 initial, 0.3280.328 peak, 0.2000.200 steady-state), indicating that usable signal is present but poorly scaled in the unshaped reward. Its likelihood advantage should not be read as evidence for latent context. The original objective includes a one-shot imitation term on the policy, and its per-decision likelihood (−1.288±0.026-1.288± 0.026 micro) sits at the pooled behavior-cloning anchor (−1.26-1.26) rather than above it. Its latent is 33-dimensional and its posterior underwent variance collapse during training. As in Section C, implementation and objective choices move every metric substantially more than latent context does. Table 12: Original PEMIRL implementation on the Arctic corpus, seeds 0,1,2\0,1,2\, evaluated under our protocol. Transfer is the steady-state goal-reaching rate of a fresh PPO agent on the frozen reward; shaped transfer adds the potential terms γV(s′)−V(s)γ V(s )-V(s). Metric Original implementation L micro ↑ −1.288± 0.026-1.288 \,±\,0.026 L macro ↑ −1.141± 0.026-1.141 \,±\,0.026 Hausdorff (km) ↓ 176.6± 15.4176.6 \,±\,15.4 Reached goal ↑ 0.53± 0.020.53 \,±\,0.02 FEE ↓ 356± 15356 \,±\,15 Transfer, raw ↑ 0.35± 0.240.35 \,±\,0.24 Transfer, shaped ↑ 0.43± 0.210.43 \,±\,0.21 Operational observations. A run counts as diverged if the discriminator loss reaches 0 or exceeds 10310^3, if the mean recovered reward exceeds 10310^3 in magnitude, if any loss is NaN, or if the final test likelihood falls below the masked-uniform baseline. No seed tripped these criteria at the end of training, but three operational issues were observed: • Reward magnitude drift: The reward output magnitude grew unconstrained (reaching order 10310^3). • Numerical loss spikes: One seed experienced an infinite discriminator loss for a single iteration before recovering. • Context variance collapse: The posterior context encoder exhibited variance collapse across training episodes. Unbounded reward scaling is consistent with the poor transfer performance observed with frozen rewards from this implementation. Appendix J J. Semi-Synthetic Diagnostics We use semi-synthetic experiments to test whether the evaluation pipeline can detect known latent variation. These results apply only to the constructed settings and the per-decision log-likelihood metric. Corridor preferences. We create two latent navigation styles by adding ±λ±λ times normalized latitude to the MCE-IRL reward. We test λ∈0,0.25,0.5,1.0,2.0λ∈\0,0.25,0.5,1.0,2.0\. Each setting contains about 960 episodes from 40 synthetic vessels, with 30 vessels for training and 10 for testing. AIRL uses no latent context; PEMIRL uses an 8-dimensional context. As λ increases, the style effect size increases from η2=0.003η^2=0.003 to 0.3210.321 (Table 13). However, AIRL outperforms PEMIRL at every dose. At λ=2.0λ=2.0, an oracle policy with true style labels improves over pooled behavior cloning by only 0.0310.031 nats per decision. Thus, corridor preferences create limited headroom under this metric because they change decisions at relatively few graph locations. Table 13: Semi-synthetic corridor-preference results. Values are held-out per-decision log-likelihood; higher is better. Gap is PEMIRL minus AIRL. Dose λ η2η^2 AIRL PEMIRL Gap 0.000.00 0.0030.003 −1.342-1.342 −1.435-1.435 −0.093-0.093 0.250.25 0.0390.039 −1.308-1.308 −1.394-1.394 −0.086-0.086 0.500.50 0.1180.118 −1.285-1.285 −1.447-1.447 −0.162-0.162 1.001.00 0.2340.234 −1.364-1.364 −1.435-1.435 −0.071-0.071 2.002.00 0.3210.321 −1.316-1.316 −1.420-1.420 −0.104-0.104 Directional habits. We also create latent styles by adding action bonuses to alternating compass directions. At the largest dose, an oracle with true style labels has 0.1330.133 nats of headroom over a shared policy. AIRL achieves −1.480-1.480 nats per decision, while PEMIRL achieves −1.600-1.600. The PEMIRL posterior does not separate the two styles: a style-classification probe is at chance accuracy (0.5000.500). Increasing the information weight to η=0.1η=0.1 improves PEMIRL by approximately 0.060.06 nats, but it remains below AIRL. These experiments use one seed and ten test vessels. They show that, in this MDP, the tested forms of latent variation provide limited per-decision likelihood headroom, and the evaluated PEMIRL posterior does not recover that available signal. References J. Chen, D. Tamboli, T. Lan, and V. Aggarwal (2023) Multi-task hierarchical adversarial inverse reinforcement learning. In Proc. ICML, PMLR, Vol. 202, p. 4895–4920. Cited by: Appendix H. J. Fu, K. Luo, and S. Levine (2018) Learning robust rewards with adversarial inverse reinforcement learning. In Proc. ICLR, Cited by: Appendix H. J. Ho and S. Ermon (2016) Generative adversarial imitation learning. In Adv. Neural Inf. Process. Syst., Vol. 29, p. 4565–4573. Cited by: Appendix H. S. Olejnik and J. Algina (2003) Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol. Methods 8 (4), p. 434–447. External Links: Document Cited by: Appendix F. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. Note: arXiv:1707.06347 External Links: Document Cited by: Appendix H. L. Yu, T. Yu, C. Finn, and S. Ermon (2019) Meta-inverse reinforcement learning with probabilistic context variables. In Adv. Neural Inf. Process. Syst., Vol. 32, p. 11772–11783. Cited by: Appendix H, Appendix I. B. D. Ziebart, J. A. Bagnell, and A. K. Dey (2010) Modeling interaction via the principle of maximum causal entropy. In Proc. ICML, p. 1255–1262. Cited by: Appendix H.