Paper deep dive
Identifying Latent Actions and Dynamics from Offline Data via Demonstrator Diversity
Felix Schur
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/22/2026, 5:55:22 AM
Summary
The paper introduces a framework for identifying latent actions and environment dynamics from action-free offline trajectories by leveraging demonstrator diversity. By assuming that demonstrator identity influences the next observation only through a latent action, the authors show that the conditional next-observation distribution can be decomposed into a mixture of latent action-conditioned transition kernels. They prove that under conditions of sufficient policy diversity and rank, these latent factors are identifiable up to a permutation of action labels, which can be resolved using minimal labeled data.
Entities (5)
Relation Signals (3)
Demonstrator Identity → affects → Next Observation
confidence 98% · identity affects the next observation only through the chosen action.
Demonstrator Diversity → enables → Identifiability of Latent Actions
confidence 95% · These results establish demonstrator diversity as a principled source of identifiability for learning latent actions and dynamics from offline RL data.
Nonnegative Matrix Factorization → models → Observable Conditional Distribution
confidence 92% · We show that this induces, for each state, a column-stochastic nonnegative matrix factorization of the observable conditional distribution.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Can latent actions and environment dynamics be recovered from offline trajectories when actions are never observed? We study this question in a setting where trajectories are action-free but tagged with demonstrator identity. We assume that each demonstrator follows a distinct policy, while the environment dynamics are shared across demonstrators and identity affects the next observation only through the chosen action. Under these assumptions, the conditional next-observation distribution $p(o_{t+1}\mid o_t,e)$ is a mixture of latent action-conditioned transition kernels with demonstrator-specific mixing weights. We show that this induces, for each state, a column-stochastic nonnegative matrix factorization of the observable conditional distribution. Using sufficiently scattered policy diversity and rank conditions, we prove that the latent transitions and demonstrator policies are identifiable up to permutation of the latent action labels. We extend the result to continuous observation spaces via a Gram-determinant minimum-volume criterion, and show that continuity of the transition map over a connected state space upgrades local permutation ambiguities to a single global permutation. A small amount of labeled action data then suffices to fix this final ambiguity. These results establish demonstrator diversity as a principled source of identifiability for learning latent actions and dynamics from offline RL data.
Tags
Links
- Source: https://arxiv.org/abs/2603.17577v1
- Canonical: https://arxiv.org/abs/2603.17577v1
Trouble viewing inline? Open PDF directly →
Full Text
61,071 characters extracted from source content.
Expand or collapse full text
Identifying Latent Actions and Dynamics from Offline Data via Demonstrator Diversity Felix Schur111felix.schur@stat.math.ethz.ch Department of Mathematics ETH Zürich Abstract Can latent actions and environment dynamics be recovered from offline trajectories when actions are never observed? We study this question in a setting where trajectories are action-free but tagged with demonstrator identity. We assume that each demonstrator follows a distinct policy, while the environment dynamics are shared across demonstrators and identity affects the next observation only through the chosen action. Under these assumptions, the conditional next-observation distribution p(ot+1∣ot,e)p(o_t+1 o_t,e) is a mixture of latent action-conditioned transition kernels with demonstrator-specific mixing weights. We show that this induces, for each state, a column-stochastic nonnegative matrix factorization of the observable conditional distribution. Using sufficiently scattered policy diversity and rank conditions, we prove that the latent transitions and demonstrator policies are identifiable up to permutation of the latent action labels. We extend the result to continuous observation spaces via a Gram-determinant minimum-volume criterion, and show that continuity of the transition map over a connected state space upgrades local permutation ambiguities to a single global permutation. A small amount of labeled action data then suffices to fix this final ambiguity. These results establish demonstrator diversity as a principled source of identifiability for learning latent actions and dynamics from offline RL data. 1 Introduction Recent progress in machine learning has been driven in large part by the availability of massive internet-scale datasets. Large language models can be trained on web text because both the input and the prediction target are directly observed. Reinforcement learning, by contrast, has not benefited from web-scale pretraining in the same way. Although the internet contains enormous amounts of sequential data—gameplay videos, robot videos, screen recordings, and human demonstrations—these data typically do not come with the action labels needed to train a policy or a dynamics model in the standard way (Seo et al., 2022; Baker et al., 2022). This lack of action annotations is a central obstacle to scaling RL from passive data. A more fundamental difficulty is that, from observation-only trajectories alone, it is generally impossible to tell whether a change from oto_t to ot+1o_t+1 was caused by an action or whether it is simply a random event in the environment. As a simple example, imagine a video of a television screen whose channel changes over time. From the video alone, we cannot tell whether the TV is broken and changes channels by itself, or whether an unseen agent is using a remote control. Both mechanisms can produce the same observed transitions. More generally, if we only observe the marginal conditional distribution p(ot+1∣ot)p(o_t+1 o_t) under a single behavior policy, then action choice and environment stochasticity are confounded: many different latent action spaces, policies, and transition kernels can induce exactly the same observed next-observation distribution. Thus, latent actions and action-conditioned dynamics are not identifiable from a single stream of observation-only data without additional structure. We propose to resolve this ambiguity by exploiting demonstrator diversity. The intuition is again simple. If a channel switch happens with the same probability for all viewers, then it is natural to attribute it to the TV itself rather than to the viewer. But if the transition probabilities differ systematically across viewers, then this variation must be mediated by action choice, provided that viewer identity has no direct effect on the TV beyond the chosen action. In other words, demonstrator identity induces variation in the latent policy while the transition dynamics remain shared. If the demonstrator policies are sufficiently diverse, this variation is rich enough to make the latent actions and dynamics identifiable. We formalize this idea in a setting where the action ata_t is unobserved, but the demonstrator identity e is observed together with (ot,ot+1)(o_t,o_t+1). We assume that each demonstrator follows a potentially different policy πe(a∣o) _e(a o), while the environment dynamics p(o′∣o,a)p(o o,a) are shared across demonstrators and identity affects the next observation only through the action. Under these assumptions, for each fixed o the observable conditional distribution p(⋅∣o,e)p(· o,e) admits a mixture decomposition over latent actions, p(⋅∣o,e)=∑a=1kp(⋅∣o,a)πe(a∣o).p(· o,e)= _a=1^kp(· o,a)\, _e(a o). This yields a column-stochastic factorization of the observable conditionals into latent action-conditioned transition kernels and demonstrator-specific action probabilities. Our main result shows that when the demonstrator policies are sufficiently diverse, this factorization becomes identifiable up to a permutation of the latent action labels. In the finite case, we cast the problem as a column-stochastic nonnegative matrix factorization and leverage minimum-volume identifiability results under a sufficiently scattered condition (Fu et al., 2018; Huang et al., 2018). In the continuous case, we derive an analogous result based on minimizing the Gram determinant of the latent transition functions. We further show that if the transition map varies continuously over a connected observation space, then the state-wise permutation ambiguity cannot vary across observations and must therefore be global. Finally, a small amount of labeled action data is enough to fix this remaining symmetry completely. More broadly, our results suggest a route toward scaling RL from passive sequential data: rather than treating heterogeneous demonstrators as nuisance variation, we can use their differences as a source of information. Demonstrator diversity makes it possible to distinguish what is truly random in the world from what is driven by latent decisions, and thereby to recover action semantics and dynamics from action-free offline data. 2 Related Work A large body of work studies how to learn from demonstrations that do not contain action labels. Early imitation from observation methods such as Behavioral Cloning from Observation (BCO) and Generative Adversarial Imitation from Observation (GAIfO) learn directly from state-only demonstrations, typically by fitting an inverse dynamics model or by matching state-transition occupancy measures (Torabi et al., 2018, 2019). Closer to our setting, ILPO learns a discrete latent action space from observations alone and then aligns the learned latent actions with real environment actions using a small amount of interaction (Edwards et al., 2019). LAPO, which learns latent action policies and world models from videos and shows that meaningful action structure can often be recovered from dynamics alone (Schmidt and Jiang, 2024). Our work is complementary to these approaches: rather than proposing a new empirical objective for latent action discovery, we ask when latent actions and action-conditioned dynamics are identifiable in principle from (ot,ot+1,e)(o_t,o_t+1,e). Several recent works aim to leverage the abundance of action-free trajectories while using only a small amount of labeled data. APV studies representation learning from action-free videos for downstream RL (Seo et al., 2022), while VPT uses a small amount of labeled gameplay to train an inverse dynamics model that labels large-scale internet videos for policy pretraining (Baker et al., 2022). Semi-supervised offline RL formalizes datasets that mix action-labeled and action-free trajectories and studies practical pipelines that infer missing actions before applying standard offline RL algorithms (Zheng et al., 2023). Our setting is more restrictive in one sense and more ambitious in another: we do not assume access to an external action labeler for the main identification argument, and instead exploit demonstrator identity as a source of structure that can render the latent action model identifiable. Our use of demonstrator identity is related to recent work showing that heterogeneous datasets can make otherwise impossible action-free learning problems tractable. Most relevant is CRAFT, which studies action-free offline learning from multiple agents with differing policies in Ex-BMDPs and shows that differences across datasets can enable recovery of controllable latent structure (Levine et al., 2025). While that work focuses on representation learning of controllable state features, our goal is different: we aim to identify a discrete latent action alphabet together with the corresponding action-conditioned transition kernels. In this sense, we use demonstrator diversity not only to learn useful representations, but to identify the semantics of the hidden decisions that generated the data. Our perspective is also closely related to ideas from causality. At a high level, demonstrator identity acts as a source of exogenous policy variation: it changes the distribution over actions while, under our exclusion assumption, affecting the next observation only through the chosen action. This is reminiscent of instrumental-variable reasoning and of invariant prediction across environments (Hartford et al., 2017; Arjovsky et al., 2019). Recent work has begun to import such ideas into offline RL, for example to identify confounded transition dynamics when a valid instrument is available (Chen et al., 2022; Liao et al., 2024). Our setting differs in that the treatment itself—the action—is latent, and identification proceeds through a structured mixture decomposition rather than a standard regression problem. Technically, our results build on the literature on identifiable nonnegative matrix factorization. In particular, minimum-volume criteria together with sufficiently scattered conditions can identify stochastic factorizations up to permutation (Fu et al., 2018). Related factorization ideas have also been used to establish identifiability in latent-variable models such as hidden Markov models (Huang et al., 2018). We adapt these tools to the RL setting by interpreting the observable next-observation distributions as mixtures over latent action-conditioned transitions, indexed by demonstrator identity. This connection yields a clean identifiability theory for latent actions and dynamics from offline, action-free trajectories. 3 Problem Setting We study a Markovian environment with latent discrete actions. Let Ot∈O_t denote the observation at time t, where O is a measurable space, and let At∈≔[k]=1,…,kA_t [k]=\1,…,k\ denote an unobserved action, where the number of latent actions k is assumed known. Each trajectory is generated by one of m demonstrators, indexed by a random variable E∈[m]E∈[m]. Given the current observation Ot=oO_t=o and demonstrator identity E=eE=e, the demonstrator chooses a latent action according to a policy At∼πe∗(⋅∣o),A_t _e^*(· o), and the environment evolves according to an action-conditioned transition kernel Ot+1∼p∗(⋅∣o,At).O_t+1 p^*(· o,A_t). Actions are never observed. The offline data therefore consist only of tuples (ot,ot+1,e),(o_t,o_t+1,e), drawn from trajectories generated by multiple demonstrators. We visualize this setting in Figure 1. OtO_tOt+1O_t+1AtA_tE Figure 1: Graphical model. Demonstrator identity E affects the next observation Ot+1O_t+1 only through the latent action AtA_t, while the current observation OtO_t affects both action choice and the next observation. Observable and latent objects. At the population level, the observable object is the family of conditional distributions p∗(⋅∣o,e):o∈,e∈[m].\p^*(· o,e):o ,\ e∈[m]\. The latent objects are the action-conditioned transition kernels and the demonstrator-specific policies p∗(⋅∣o,a):o∈,a∈[k]\p^*(· o,a):o ,\ a∈[k]\ and πe∗(a∣o):e∈[m],a∈[k],o∈.\ _e^*(a o):e∈[m],\ a∈[k],\ o \. By marginalizing over the latent action, for every o∈o and e∈[m]e∈[m], p∗(⋅∣o,e)=∑a=1kp∗(⋅∣o,a)πe∗(a∣o).p^*(· o,e)= _a=1^kp^*(· o,a)\, _e^*(a o). (1) Thus, for each fixed observation o, the demonstrator-conditioned next-observation law is a mixture of latent action-conditioned transition kernels, with mixing weights given by the demonstrator-specific policy. Goal. Our goal is to recover the latent transition kernel and the demonstrator-specific policies p∗(⋅∣o,a),πe∗(a∣o)p^*(· o,a), _e^*(a o) from the observable family p∗(⋅∣o,e)\p^*(· o,e)\. Since the labels of the latent actions are arbitrary, recovery can at best hold up to a permutation of the action indices. We now state the assumptions used throughout. Assumption 1 (Structural assumptions). Assume the following. (i) Sufficiency / Markov property: Ot+1⟂(Ot−1,At−1,Ot−2,At−2,…)∣(Ot,At).O_t+1 \!\!\! (O_t-1,A_t-1,O_t-2,A_t-2,…) (O_t,A_t). (i) Exclusion restriction: E⟂Ot+1∣(Ot,At),E \!\!\! O_t+1 (O_t,A_t), that is, demonstrator identity affects the next observation only through the latent action. (i) Well-defined latent transitions: For every o∈o and every a∈[k]a∈[k] such that πe∗(a∣o)>0for some e∈[m], _e^*(a o)>0 some e∈[m], the conditional distribution p∗(⋅∣o,a)p^*(· o,a) is well-defined. 4 Theoretical Guarantees Our analysis proceeds in four steps. First, we show that without demonstrator diversity the problem is non-identifiable in general. Second, we establish statewise identifiability in finite observation spaces using identifiable stochastic (non-negative matrix factorization) NMF. Third, we give a extension to continuous observation spaces. Fourth, we show that continuity and a no-collision condition upgrade statewise permutations to a single global permutation, which can then be fixed by a small amount of labeled action data. We begin by formalizing the fact that observation-only data from a single behavior policy are not sufficient to identify latent actions and dynamics. Proposition 4.1 (Non-identifiability with a single demonstrator). Fix o∈o . Suppose only the marginal next-observation law p∗(⋅∣o)p^*(· o) is observed, without demonstrator identity. Then in general the decomposition p∗(⋅∣o)=∑a=1kp∗(⋅∣o,a)π∗(a∣o)p^*(· o)= _a=1^kp^*(· o,a)\,π^*(a o) is not identifiable: there exist distinct pairs of latent transitions and action probabilities, not related by permutation, that induce the same observable distribution. Proposition 4.1 shows that additional structure is necessary. In our setting, demonstrator identity provides this structure by inducing observable variation in action probabilities while keeping the transition kernel fixed. 4.1 Statewise identifiability in finite observation spaces We first consider the case where O is finite. Fix o∈o . For each demonstrator e∈[m]e∈[m], define peo≔p∗(⋅∣o,e)∈ℝ||,p_e^o p^*(· o,e) ^|O|, and tao≔p∗(⋅∣o,a)∈ℝ||,t_a^o p^*(· o,a) ^|O|, and collect these into the matrices Po∗≔[p1o⋯pmo]∈ℝ||×m,P_o^* bmatrixp_1^o&·s&p_m^o bmatrix ^|O|× m,, To∗≔[t1o⋯tko]∈ℝ||×k,T_o^* bmatrixt_1^o&·s&t_k^o bmatrix ^|O|× k, and Πo∗≔[π1∗(⋅∣o)⋯πm∗(⋅∣o)]∈ℝk×m. _o^* bmatrix _1^*(· o)&·s& _m^*(· o) bmatrix ^k× m. By (1), Po∗=To∗Πo∗.P_o^*=T_o^* _o^*. (2) Each column of Po∗P_o^*, To∗T_o^*, and Πo∗ _o^* is a probability vector. Define the set of column-stochastic matrices ℳr×c≔M∈[0,1]r×c| 1⊤M=⊤.M_r× c \M∈[0,1]^r× c\; |\; 1 M= 1 \. Then Po∗∈ℳ||×m,P_o^* _|O|× m, To∗∈ℳ||×k,T_o^* _|O|× k, and Πo∗∈ℳk×m. _o^* _k× m. The finite-state identification problem is therefore: from the observable matrix Po∗P_o^*, recover the latent factors (To∗,Πo∗)(T_o^*, _o^*) up to permutation. The factorization (2) is not unique in general: if S∈GL(k)S (k) is invertible, then Po∗=To∗Πo∗=(To∗S)(S−1Πo∗).P_o^*=T_o^* _o^*=(T_o^*S)(S^-1 _o^*). Thus additional structure is needed to identify the latent factors. We use the sufficiently scattered condition of Fu et al. (2018), which ensures that among all feasible stochastic factorizations, the true one is the unique minimum-volume solution up to permutation. Definition 4.1 (Sufficiently scattered). Let M∈ℝu×vM ^u× v be nonnegative. Define cone(M)≔Mθ:θ≥0cone(M) \Mθ:θ≥ 0\ and C∗≔y:y⊤x≥0 for all x∈CC^* \y:y x≥ 0 for all x∈ C\ for any cone C. For v≥2v≥ 2, define the second-order cones v≔x∈ℝv:⊤x≥v−1‖x‖2C_v \x ^v: 1 x≥ v-1\,\|x\|_2 \ and v∗≔x∈ℝv:⊤x≥‖x‖2.C_v^* \x ^v: 1 x≥\|x\|_2 \. A matrix H∈ℳu×vH _u× v is called sufficiently scattered if v⊆cone(H⊤)andcone(H⊤)∗∩bd(v∗)=λej:j∈[v],λ≥0.C_v (H ) (H )^* (C_v^*)=\λ e_j:j∈[v],\ λ≥ 0\. Assumption 2 (Finite-state identifiability conditions). For all o∈o we have ||≥k|O|≥ k and m≥km≥ k and (i) rank(Po∗)=krank(P_o^*)=k; (i) Πo∗ _o^* is sufficiently scattered. Intuitively, the sufficiently scattered condition means that the demonstrator policy vectors are not all small perturbations of one another or concentrated near a low-dimensional subset of the simplex; rather, they point in sufficiently many different action-mixture directions that their conic hull spreads broadly through the positive orthant, making the latent actions geometrically separable. In this sense it formalizes policy diversity: different demonstrators must place meaningfully different relative masses on the latent actions, so that each latent action leaves a distinct signature in the observed mixtures instead of being confounded with the others. We can now state the finite-state identifiability result. Theorem 4.1 (Statewise identifiability in finite observation spaces). Assume 2. Then for every o∈o , the optimization problem minTo∈ℳ||×k,Πo∈ℳk×m _T_o _|O|× k,\, _o _k× m det(To⊤To) (T_o T_o) (3) s.t. Po∗=ToΠo P_o^*=T_o _o recovers the latent factors up to a permutation of the action labels. More precisely, there exists a permutation matrix Σo∈0,1k×k _o∈\0,1\^k× k such that To=To∗Σo,Πo=Σo⊤Πo∗.T_o=T_o^* _o, _o= _o _o^*. The minimum-determinant objective admits a simple geometric interpretation. Since det(To⊤To) (T_o T_o) is the squared k-dimensional volume spanned by the columns of ToT_o, minimizing it selects, among all stochastic factorizations of Po∗P_o^*, the latent transition kernels that are least spread out while still explaining the demonstrator-conditioned transition laws. Intuitively, this is a conservative principle: we attribute to the latent actions only the variation that is forced by the observed diversity across demonstrators, and do not introduce more extreme latent action effects than necessary. If the demonstrators are sufficiently diverse, this still forces recovery of all action effects that are genuinely distinguishable from the data. On the other hand, if two latent actions induce the same transition law at state o, then they are observationally equivalent there and cannot be identified separately; in that case the true factorization becomes rank-deficient (rank(Po∗)<krank(P_o^*)<k) and det((To∗)⊤To∗)=0 ((T_o^*) T_o^*)=0. Nevertheless, if one reduces the problem to the effective dimension ro≔rank(Po∗),r_o (P_o^*), which under sufficient policy diversity coincides with the number of distinct transition effects at o, then the same minimum-volume argument identifies the remaining distinct latent transitions (and the corresponding aggregated policy masses) up to permutation. Theorem 4.1 is a statewise result: the permutation Σo _o may still depend on o. The next subsections address both continuous observation spaces and alignment of these statewise permutations across o. 4.2 Continuous-space identifiability via embedded Gram-determinant We now give a continuous-state analogue of the finite-state minimum-volume result that applies to general transition measures, including deterministic transitions represented by Dirac masses. Let (,ℬ)(O,B) be a measurable space, let ()P(O) denote the set of probability measures on (,ℬ)(O,B), and let ℳ()M(O) denote the vector space of finite signed measures on (,ℬ)(O,B). Fix a real Hilbert space ℋH and a linear injective map Φ:ℳ()→ℋ. :M(O) . A canonical example is a kernel mean embedding into an RKHS associated with a bounded characteristic kernel. Injectivity ensures that equality in embedding space implies equality of the underlying measures. Fix o∈o . Define the latent transition measures tao≔p∗(⋅∣o,a)∈(),a∈[k],t_a^o p^*(· o,a) (O),a∈[k], and the observable demonstrator-conditioned transition measures peo≔p∗(⋅∣o,e)∈(),e∈[m].p^o_e p^*(· o,e) (O),e∈[m]. By the mixture identity, peo=∑a=1kπe∗(a)tao,e∈[m].p^o_e= _a=1^k _e^*(a)\,t_a^o,e∈[m]. Applying the linear map Φ yields Φ(peo)=∑a=1kπe∗(a)Φ(tao). (p^o_e)= _a=1^k _e^*(a)\, (t_a^o). For any k-tuple of measures (t1,…,tk)(t_1,…,t_k), define the embedded Gram matrix GΦ(t1,…,tk)≔[⟨Φ(ti),Φ(tj)⟩ℋ]i,j=1k.G_ (t_1,…,t_k) [ (t_i), (t_j) _H ]_i,j=1^k. Its determinant measures the squared volume spanned by the embedded latent transition measures in ℋH. We also define the rank of a family of embedded observables by rank(P∗)≔dimspanΦ(p1o),…,Φ(pmo)rank(P^*) \ (p^o_1),…, (p^o_m)\. Note that Πo∗=[π1∗(⋅∣o)⋯πm∗(⋅∣o)]∈ℝk×m _o^*= bmatrix _1^*(· o)&·s& _m^*(· o) bmatrix ^k× m remains a real-valued matrix and therefore the sufficiently scattered condition does not change. Assumption 3 (Continuous-state identifiability conditions). For all o∈o we have ||≥k|O|≥ k and m≥km≥ k and (i) rank(GΦ(t1o,…,tko))=krank (G_ (t^o_1,…,t^o_k) )=k; (i) Πo∗ _o^* is sufficiently scattered. We now state the continuous-state identifiability result. Theorem 4.2 (State-wise identifiability in continuous observation spaces). Fix o∈o and suppress the dependence on o in the notation. Assume 3 and that the optimization problem mint¯1,…,t¯k,Π _ t_1,…, t_k, det(GΦ(t¯1,…,t¯k)) (G_ ( t_1,…, t_k) ) (4) s.t. peo=∑a=1kπe(a)t¯a,e∈[m], p^o_e= _a=1^k _e(a) t_a, e∈[m], Πo∈ℳk×m, _o _k× m, t¯a∈(),a∈[k], t_a (O), a∈[k], admits an optimizer. Then every optimizer of (4) is equal to the ground-truth factorization up to permutation: there exists a permutation matrix Σo∈0,1k×k _o∈\0,1\^k× k such that t¯a=tΣ(a)o,a∈[k], t_a=t^o_ (a),a∈[k], and Πo=Σo⊤Πo∗. _o= _o _o^*. In finite-dimensional stochastic NMF, optimizer existence is often automatic by compactness. In contrast, (4) is an optimization problem over an infinite-dimensional space of measures, so minimizing sequences need not converge without additional compactness assumptions. A standard sufficient route is to work on a compact Polish observation space and choose Φ to be weakly continuous (for example, a kernel mean embedding associated with a bounded continuous kernel). Then ()P(O) is weakly compact, ℳk×mM_k× m is compact, the feasibility constraints are closed, and the objective (t1,…,tk)↦det(GΦ(t1,…,tk))(t_1,…,t_k) (G_ (t_1,…,t_k) ) is continuous, so an optimizer exists by the direct method. 4.2.1 From state-wise to global identifiability The state-wise identifiability results identify the latent action model separately at each observation o, but only up to an observation-dependent permutation. We now show that, under continuity and connectedness, these local permutations must agree globally. Throughout this subsection, let Φ:ℳ()→ℋ :M(O) be the fixed injective linear embedding used in the continuous-space theorem, where ℋH is a real Hilbert space. For each o∈o and a∈[k]a∈[k], define the embedded true latent transitions ϕa∗(o)≔Φ(p∗(⋅∣o,a))∈ℋ, _a^*(o) \! (p^*(· o,a) ) , and, for any candidate latent model, define the embedded recovered latent transitions ϕa(o)≔Φ(p(⋅∣o,a))∈ℋ. _a(o) \! (p(· o,a) ) . We collect these into the operator-valued maps T∗(o)≔[ϕ1∗(o)⋯ϕk∗(o)],T(o)≔[ϕ1(o)⋯ϕk(o)].T^*(o) bmatrix _1^*(o)&·s& _k^*(o) bmatrix, T(o) bmatrix _1(o)&·s& _k(o) bmatrix. Assumption 4. Assume that: (i) for each a∈[k]a∈[k], the maps o↦ϕa(o)o _a(o) and o↦ϕa∗(o)o _a^*(o) are continuous from O into ℋH; (i) for every o∈o and every i≠ji≠ j, ϕi∗(o)≠ϕj∗(o). _i^*(o)≠ _j^*(o). Theorem 4.3 (From local to global permutations). Assume that O is a connected metric space. Assume 4. Assume that for each o∈o there exists a permutation matrix Σ(o) (o) such that T(o)=T∗(o)Σ(o),Π(o)=Σ(o)⊤Π∗(o).T(o)=T^*(o) (o), (o)= (o) ^*(o). Then Σ(o) (o) is constant on O. That is, there exists a single permutation matrix Σ such that T(o)=T∗(o)Σ,Π(o)=Σ⊤Π∗(o)for all o∈.T(o)=T^*(o) , (o)= ^*(o) all o . Theorem 4.3 shows that once the latent model is identified at each observation up to permutation, continuity and connectedness reduce the ambiguity to a single global permutation of the latent action labels. We now combine the state-wise identifiability theorem with Theorem 4.3 to obtain global identifiability. Assumption 5 (Global identifiability conditions). Assume the following. (i) O is a connected metric space. (i) For every o∈o , the policy matrix Π∗(o)∈ℳk×m ^*(o) _k× m is sufficiently scattered. (i) For every o∈o , rank(GΦ(t1o,…,tko))=krank (G_ (t^o_1,…,t^o_k) )=k. (iv) For each a∈[k]a∈[k], the map o↦Φ(p∗(⋅∣o,a))o \! (p^*(· o,a) ) is continuous from O into ℋH. (v) There exists a nonempty subset S⊆S on which the global permutation is known. We give some intuition about the 5: Connectedness rules out choosing different action labelings on disconnected regions of the observation space and continuity prevents the local permutation ambiguity from changing abruptly across nearby observations (see Theorem 4.3). The sufficient-scattering condition guarantees state-wise identifiability by making the demonstrator policies diverse enough to separate the latent action-conditioned transition kernels and the positive-definiteness of the embedded Gram matrix ensures that the latent action effects are linearly independent, and hence distinguishable, at each observation (see Theorem 4.2). Corollary 4.1 (Global identifiability). Assume 5(i)-(iv) hold. Let o↦(p(⋅∣o,1),…,p(⋅∣o,k),Π(o))o (p(· o,1),…,p(· o,k), (o) ) be any continuous family such that for every o∈o : (a) the tuple (p(⋅∣o,1),…,p(⋅∣o,k),Π(o)) (p(· o,1),…,p(· o,k), (o) ) is feasible for (4), and (b) it attains the minimum in (4). Then there exists a single permutation matrix Σ such that p(⋅∣o,a)=p∗(⋅∣o,Σ(a)),Π(o)=Σ⊤Π∗(o)for all o∈.p(· o,a)=p^*(· o, (a)), (o)= ^*(o) all o . If, in addition, 5(v) holds, then Σ=Ik =I_k, and therefore p(⋅∣o,a)=p∗(⋅∣o,a),Π(o)=Π∗(o)for all o∈,a∈[k].p(· o,a)=p^*(· o,a), (o)= ^*(o) all o ,\ a∈[k]. In particular, the latent action-conditioned transition kernels and the demonstrator-specific policies are globally identifiable. Although the assumptions in 5 are natural, one may object that in applications they need not hold uniformly over all o∈o . In particular, some assumptions may fail on a small subset of the observation space or only at isolated observations. The key point is that the identifiability result is often robust to such localized violations: even when global identifiability fails on all of O, one can typically still recover identifiability on the subset where the assumptions remain valid. More concretely, the consequences of violating the assumptions can be summarized as follows. • If 5(i) fails, then O has multiple connected components. In that case, the argument of Theorem 4.3 applies separately on each connected component, so the latent actions remain identifiable up to a component-wise permutation. If 5(v) holds on each component, then these permutations are fixed and one still obtains full identifiability on all of O. • If 5(i) fails at some observations o∈A⊊o∈ A , then the geometric condition needed for the state-wise minimum-volume argument may break down on A. At such observations, state-wise identifiability is no longer guaranteed. Nevertheless, if ∖AO A remains connected and 5(i) holds there, then the global permutation is still uniquely determined on ∖AO A. • If 5(i) fails at some o∈o , then the embedded latent transitions become linearly dependent at that observation. In particular, the action-conditioned effects are no longer fully distinguishable there, so one cannot expect unique recovery of all latent actions at that state. This is an intrinsic non-identifiability phenomenon rather than a limitation of our results. • If 5(iv) fails, then the local permutation need no longer vary continuously with o. As a result, different permutations may be selected in different regions of the observation space. However, if the recovered and true embedded transitions are continuous on each element of a partition of O, then the conclusion of Theorem 4.3 still applies piecewise, yielding one permutation per region. • If 5(v) fails, then the latent actions remain identifiable only up to a single global permutation. This is the unavoidable label-swapping ambiguity familiar from latent-variable models. 5 Estimation The identifiability results above are stated at the population level and characterize the latent model as the minimum-volume factorization of the observable conditional laws subject to exact mixture constraints and sufficient policy diversity. In practice, however, we only observe finitely many triples (ot,ot+1,et),(o_t,o_t+1,e_t), and in continuous observation spaces it is not tractable to optimize directly over arbitrary transition measures and state-dependent policy matrices. We therefore consider a parametric estimation procedure that is motivated by the identification theory but replaces the exact constrained problem by a relaxed empirical objective. We parameterize the latent action-conditioned transition model and the demonstrator-specific latent policy by neural networks pθ(⋅∣o,a),πψ(a∣o,e),p_θ(· o,a), _ψ(a o,e), where θ and ψ denote trainable parameters. The policy network outputs a probability vector over the k latent actions, for example via a softmax layer, and the transition model outputs a conditional distribution over next observations. In continuous spaces, pθ(⋅∣o,a)p_θ(· o,a) may be instantiated as a Gaussian head, a mixture density model, or a conditional latent-state model. The observable next-observation law induced by the model is pθ,ψ(o′∣o,e)=∑a=1kπψ(a∣o,e)pθ(o′∣o,a),p_θ,ψ(o o,e)= _a=1^k _ψ(a o,e)\,p_θ(o o,a), (5) which mirrors the population mixture representation in (1). A natural estimation principle is maximum likelihood under the observable mixture model (5). Given a dataset =(oi,oi′,ei)i=1n,D=\(o_i,o_i ,e_i)\_i=1^n, we define the empirical negative log-likelihood ℒfit(θ,ψ)=−1n∑i=1nlog(∑a=1kπψ(a∣oi,ei)pθ(oi′∣oi,a)).L_fit(θ,ψ)=- 1n _i=1^n ( _a=1^k _ψ(a o_i,e_i)\,p_θ(o_i o_i,a) ). Minimizing ℒfitL_fit alone is generally insufficient: the model can explain the data with collapsed latent actions or nearly identical demonstrator policies, even when the true latent structure is identifiable. A minimum-volume regularizer for latent transitions. The continuous-space identifiability result suggests selecting, among all factorizations that explain the data well, the one with minimum embedded volume. Let Φ:ℳ()→ℋ :M(O) be the injective linear embedding used in Theorem 4.2. For a candidate transition model, define the embedded latent transitions μθ,a(o)≔Φ(pθ(⋅∣o,a))∈ℋ,a∈[k]. _θ,a(o) (p_θ(· o,a) ) , a∈[k]. Their embedded Gram matrix is Gθ(o)≔[⟨μθ,a(o),μθ,b(o)⟩ℋ]a,b=1k.G_θ(o) [ _θ,a(o), _θ,b(o) _H ]_a,b=1^k. A practical surrogate for the population minimum-volume principle is then ℛvol(θ)=1n∑i=1nlogdet(Gθ(oi)+εIk),R_vol(θ)= 1n _i=1^n \! (G_θ(o_i)+ I_k ), where ε>0 >0 is a small numerical regularization constant. Minimizing ℛvolR_vol encourages the latent transition family to be as simple and compact as possible while still explaining the observed data. Preventing policy collapse. In the identifiability theory, demonstrator diversity is not an objective to be maximized, but a condition on the feasible factorizations: the true policy matrix must be sufficiently scattered. A direct practical analogue is therefore not to reward arbitrarily large diversity, but to penalize degenerate policy geometries in which the learned policy matrix becomes nearly low-rank or some latent actions become unused. For a fixed observation o, define the learned policy matrix Πψ(o)≔[πψ(⋅∣o,1)⋯πψ(⋅∣o,m)]∈ℝk×m. _ψ(o) bmatrix _ψ(· o,1)&·s& _ψ(· o,m) bmatrix ^k× m. To discourage collapse, we introduce a diversity barrier based on the log-determinant of the policy Gram matrix: ℛpol(ψ)=1n∑i=1n[τ−logdet(Πψ(oi)Πψ(oi)⊤+εIk)]+,R_pol(ψ)= 1n _i=1^n [τ- \! ( _ψ(o_i) _ψ(o_i) + I_k ) ]_+, where [x]+=maxx,0[x]_+= \x,0\ and τ is a user-specified threshold. This term vanishes whenever the policy matrix is sufficiently well-conditioned, and becomes active only when demonstrator policies begin to collapse toward a low-dimensional or poorly separated configuration. In this way, it acts as a soft feasibility constraint rather than a competing objective. Optional label anchoring. As in the theoretical development, the latent action labels are only identifiable up to permutation unless a small amount of side information is available. If a small labeled dataset lab=(oj,ej,aj⋆)j=1rD_lab=\(o_j,e_j,a_j )\_j=1^r is available, then the residual global permutation ambiguity can be fixed by adding the supervision term ℒanchor(ψ)=−1r∑j=1rlogπψ(aj⋆∣oj,ej).L_anchor(ψ)=- 1r _j=1^r _ψ(a_j o_j,e_j). Final objective. Combining the terms above yields the estimator minθ,ψℒfit(θ,ψ)+λvolℛvol(θ)+λpolℛpol(ψ)+λanchorℒanchor(ψ), _θ,ψ _fit(θ,ψ)+ _vol\,R_vol(θ)+ _pol\,R_pol(ψ)+ _anchor\,L_anchor(ψ), (6) where λvol,λpol,λanchor≥0 _vol, _pol, _anchor≥ 0 are tuning parameters. The roles of the terms are complementary: ℒfitL_fit enforces agreement with the observed demonstrator-conditioned transitions, ℛvolR_vol selects a simple latent action representation, ℛpolR_pol keeps the learned policies in a sufficiently diverse regime, and ℒanchorL_anchor resolves the final global label symmetry when labeled actions are available. The objective (6) should be viewed as a practical surrogate for the population identification problem rather than as an exact finite-sample analogue of the theory. In particular, the minimum-volume principle in Theorem 4.2 is applied there over exactly feasible factorizations, whereas (6) trades off data fit and regularization in finite samples. The policy regularizer is therefore important in practice: without it, the model may reduce the transition volume by collapsing distinct latent actions or by learning nearly identical demonstrator policies, even when the ground-truth factorization is identifiable. The estimator above is designed to preserve the geometric intuition of the theory: learn the simplest latent action-conditioned transition model that explains the data, while requiring the demonstrator policies to remain sufficiently diverse to reveal that latent structure. Computing the embedded Gram matrix. The form of Gθ(o)G_θ(o) depends on how the transition model pθ(⋅∣o,a)p_θ(· o,a) is represented. If pθp_θ is an explicit density model on O, such as a Gaussian or mixture-density head, and the embedding Φ is induced by a kernel k, then the Gram entries can be written as Gθ(o)ab=⟨Φ(pθ(⋅∣o,a)),Φ(pθ(⋅∣o,b))⟩ℋ=x∼pθ(⋅∣o,a),y∼pθ(⋅∣o,b)[k(x,y)],G_θ(o)_ab= (p_θ(· o,a)), (p_θ(· o,b)) _H=E_x p_θ(· o,a),\,y p_θ(· o,b)[k(x,y)], which may be available in closed form for simple choices of pθp_θ and k, or else approximated by Monte Carlo samples. If the transition model is deterministic, say pθ(⋅∣o,a)=δfθ(o,a),p_θ(· o,a)= _f_θ(o,a), then Gθ(o)ab=k(fθ(o,a),fθ(o,b)),G_θ(o)_ab=k(f_θ(o,a),f_θ(o,b)), so the Gram matrix reduces to pairwise kernel evaluations between the predicted next observations. Finally, if the transition model is defined in a learned latent state space, for example by encoding observations as z=hω(o)z=h_ω(o) and modeling pθ(z′∣z,a)p_θ(z z,a), then the same construction may be applied in latent space by replacing O with the latent space and computing the Gram matrix from the corresponding latent transition distributions. In all cases, the role of Gθ(o)G_θ(o) is the same: it quantifies how distinct the action-conditioned transitions are at observation o, and its determinant serves as a measure of the effective volume of the learned latent action simplex. 6 Conclusion We studied when latent actions and action-conditioned dynamics can be identified from offline trajectories in which actions are never observed but demonstrator identity is available. Our main insight is that demonstrator diversity can make the latent structure recoverable: when demonstrators induce sufficiently rich variation in latent action usage and the transition dynamics are shared, the observable next-observation laws admit an identifiable factorization up to permutation of the latent actions. We established this first in finite observation spaces via identifiable stochastic nonnegative matrix factorization, and then extended the result to continuous observation spaces through an embedded minimum-volume criterion. Under continuity and connectedness assumptions, the remaining statewise permutation ambiguity reduces to a single global permutation, which can be fixed with a small amount of labeled action data. Beyond the identifiability results, our analysis suggests a practical principle for learning from action-free offline data: learn the simplest latent action-conditioned transition model consistent with the data while requiring demonstrator policies to remain sufficiently diverse to reveal that latent structure. We hope these results provide a useful theoretical foundation for future algorithms that leverage heterogeneous passive data for representation learning, model learning, and offline reinforcement learning without action annotations. References Arjovsky et al. [2019] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv:1907.02893, 2019. Baker et al. [2022] B. Baker, I. Akkaya, P. Zhokhov, J. Huizinga, J. Tang, A. Ecoffet, B. Houghton, R. Sampedro, and J. Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. In Advances in Neural Information Processing Systems, 2022. Chen et al. [2022] Y. Chen, L. Xu, C. Gulcehre, T. Le Paine, A. Gretton, N. De Freitas, and A. Doucet. On instrumental variable regression for deep offline policy evaluation. Journal of Machine Learning Research, 23(302):1–40, 2022. Edwards et al. [2019] A. D. Edwards, L. Hasenclever, A. Storkey, and Y. W. Teh. Imitating latent policies from observation. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1755–1763. PMLR, 2019. Fu et al. [2018] X. Fu, K. Huang, N. D. Sidiropoulos, and W. Ma. On identifiability of nonnegative matrix factorization. IEEE Signal Processing Letters, 25(3):328–332, 2018. doi: 10.1109/LSP.2018.2794181. Hartford et al. [2017] J. Hartford, G. Lewis, K. Leyton-Brown, and M. Taddy. Deep iv: A flexible approach for counterfactual prediction. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1414–1423. PMLR, 2017. Huang et al. [2018] K. Huang, X. Fu, and N. D. Sidiropoulos. Learning hidden markov models from pairwise co-occurrences with application to topic modeling. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2068–2077. PMLR, 2018. Levine et al. [2025] A. Levine, P. Stone, and A. Zhang. Offline action-free learning of ex-bmdps by comparing diverse datasets. arXiv preprint arXiv:2503.21018, 2025. Liao et al. [2024] L. Liao, Z. Fu, Z. Yang, Y. Wang, D. Ma, M. Kolar, and Z. Wang. Instrumental variable value iteration for causal offline reinforcement learning. Journal of Machine Learning Research, 25(303):1–56, 2024. Schmidt and Jiang [2024] D. Schmidt and M. Jiang. Learning to act without actions. In International Conference on Learning Representations, 2024. Seo et al. [2022] Y. Seo, T. Yu, S. K. S. Ghasemipour, D. Schuurmans, S. Levine, O. Nachum, and K. Lee. Reinforcement learning with action-free pre-training from videos. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 19561–19584. PMLR, 2022. Torabi et al. [2018] F. Torabi, G. Warnell, and P. Stone. Behavioral cloning from observation. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pages 4950–4957. IJCAI, 2018. doi: 10.24963/ijcai.2018/687. Torabi et al. [2019] F. Torabi, G. Warnell, and P. Stone. Generative adversarial imitation from observation. In ICML Workshop on Imitation, Intent, and Interaction (I3), 2019. PMLR Workshop Proceedings. Zheng et al. [2023] Q. Zheng, M. Henaff, B. Amos, and A. Grover. Semi-supervised offline reinforcement learning with action-free trajectories. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 42339–42362. PMLR, 2023. Appendix A Proofs A.1 Proof of Proposition 4.1 It suffices to give one example. Let =1,2O=\1,2\ and k=2k=2, and consider the observable distribution p∗(⋅∣o)=[1/21/2].p^*(· o)= bmatrix1/2\\ 1/2 bmatrix. One feasible decomposition is t1=[10],t2=[01],π∗(⋅∣o)=[1/21/2],t_1= bmatrix1\\ 0 bmatrix, t_2= bmatrix0\\ 1 bmatrix, π^*(· o)= bmatrix1/2\\ 1/2 bmatrix, for which p∗(⋅∣o)=12t1+12t2.p^*(· o)= 12t_1+ 12t_2. A different feasible decomposition is t~1=[3/41/4],t~2=[1/43/4],π~∗(⋅∣o)=[1/21/2], t_1= bmatrix3/4\\ 1/4 bmatrix, t_2= bmatrix1/4\\ 3/4 bmatrix, π^*(· o)= bmatrix1/2\\ 1/2 bmatrix, for which p∗(⋅∣o)=12t~1+12t~2.p^*(· o)= 12 t_1+ 12 t_2. These two decompositions are not related by permutation. Hence the latent transitions and latent actions are not identifiable from a single observation-only conditional distribution in general. A.2 Proof of Theorem 4.1 Theorem A.1 (Theorem 1 of Fu et al. [2018]). Let u,v,z∈ℕu,v,z with u,z≥vu,z≥ v. Suppose X∗∈ℝu×z,W∗∈ℝu×v,H∗∈ℝv×zX^* ^u× z, W^* ^u× v, H^* ^v× z satisfy X∗=W∗H∗,rank(X∗)=rank(H∗)=v,X^*=W^*H^*, (X^*)=rank(H^*)=v, and assume that H∗H^* is sufficiently scattered. Then the optimization problem minW∈ℝu×v,H∈ℝv×z _W ^u× v,H ^v× z det(W⊤W) (W W) (7) s.t. X∗=WH X^*=WH H⊤=,H≥0 H 1 = 1,H≥ 0 has a unique solution up to permutation: there exists a permutation matrix Σ∈0,1v×v ∈\0,1\^v× v and a full-rank permutation matrix D∈ℝv×vD ^v× v such that W=W∗ΣD,H=D−1Σ⊤H∗.W=W^* D, H=D^-1 H^*. Lemma A.1 (Column-stochasticity removes diagonal scaling). Let T,T′∈ℳu×kT,T _u× k and Π,Π′∈ℳk×m , _k× m satisfy T′=TD,Π′=D−1ΠT =TD, =D^-1 for some invertible diagonal matrix D∈ℝk×kD ^k× k. Then D=IkD=I_k. Proof. Write D=diag(d1,…,dk)D=diag(d_1,…,d_k). Since T and T′T are column-stochastic, ⊤T′=⊤TD=⊤D=⊤. 1 T = 1 TD= 1 D= 1 . Hence da=1d_a=1 for every a∈[k]a∈[k], so D=IkD=I_k. ∎ Under Assumption 2, Theorem A.1 implies that any feasible factorization of Po∗P_o^* is unique up to permutation and diagonal scaling. Concretely, there exist a permutation matrix Σo _o and an invertible diagonal matrix DoD_o such that To=To∗ΣoDo,Πo=Do−1Σo⊤Πo∗.T_o=T_o^* _oD_o, _o=D_o^-1 _o _o^*. By Lemma A.1, column-stochasticity forces Do=IkD_o=I_k. Therefore To=To∗Σo,Πo=Σo⊤Πo∗,T_o=T_o^* _o, _o= _o _o^*, as claimed. A.3 Proof of Theorem 4.2 Lemma A.2 (Correct determinant bound). Let Π∗∈ℝ+k×m ^* _+^k× m have rank k, and define K≔cone(Π∗)⊂ℝk.K ( ^*) ^k. Assume k⊆K,K∗∩bd(k∗)=λej:j∈[k],λ≥0.C_k K, K^* (C_k^*)=\λ e_j:j∈[k],\ λ≥ 0\. Let A∈GL(k)A (k) satisfy AΠ∗≥0,A⊤=.A ^*≥ 0, A 1= 1. Then |det(A)|≤1.| (A)|≤ 1. Moreover, equality holds if and only if A is a permutation matrix. Proof. Let r1⊤,…,rk⊤r_1 ,…,r_k denote the rows of A. Since AΠ∗≥0A ^*≥ 0, for each i we have ri⊤Π∗≥0.r_i ^*≥ 0. Equivalently, for every column πj∗ _j^* of Π∗ ^*, ri⊤πj∗≥0.r_i _j^*≥ 0. Hence each row vector rir_i lies in the dual cone K∗K^*. Because k⊆KC_k K, duality of cones gives K∗⊆k∗=x∈ℝk:⊤x≥‖x‖2.K^* _k^*=\x ^k: 1 x≥\|x\|_2\. Therefore each row satisfies ‖ri‖2≤⊤ri.\|r_i\|_2≤ 1 r_i. By Hadamard’s inequality applied to the rows of A, |det(A)|≤∏i=1k‖ri‖2≤∏i=1k⊤ri.| (A)|≤ _i=1^k\|r_i\|_2≤ _i=1^k 1 r_i. Now ⊤ri=(A)i, 1 r_i=(A 1)_i, so ∏i=1k⊤ri=∏i=1k(A)i. _i=1^k 1 r_i= _i=1^k(A 1)_i. Also, ∑i=1k(A)i=⊤A=(A⊤)⊤=⊤=k. _i=1^k(A 1)_i= 1 A 1=(A 1) 1= 1 1=k. Since each (A)i≥‖ri‖2≥0(A 1)_i≥\|r_i\|_2≥ 0, AM–GM yields ∏i=1k(A)i≤1. _i=1^k(A 1)_i≤ 1. Hence |det(A)|≤1.| (A)|≤ 1. Now suppose equality holds. Then equality must hold in: (i) Hadamard’s inequality, so the rows r1,…,rkr_1,…,r_k are pairwise orthogonal; (i) the bound ‖ri‖2≤⊤ri\|r_i\|_2≤ 1 r_i, so each ri∈bd(k∗)r_i (C_k^*); (i) AM–GM, so each (A)i=1(A 1)_i=1, i.e. each row sum equals 11. Thus each row rir_i lies in K∗∩bd(k∗),K^* (C_k^*), so by assumption, ri=λieℓir_i= _ie_ _i for some λi≥0 _i≥ 0 and ℓi∈[k] _i∈[k]. Since the row sum is 11, we get λi=1 _i=1. Thus every row is a standard basis vector. Because A is invertible, these basis vectors must be distinct, so A is a permutation matrix. Conversely, any permutation matrix satisfies the assumptions and has determinant of absolute value 11. ∎ Proof of Theorem 4.2. Fix o∈o and suppress the dependence on o in the notation. Write ta≔tao,pe≔peo,Π∗≔Πo∗.t_a t_a^o, p_e p_e^o, ^* _o^*. Let (t¯1,…,t¯k,Π)( t_1,…, t_k, ) be any feasible solution of (4). Define the embedded latent and observable elements ua≔Φ(ta)∈ℋ,u¯a≔Φ(t¯a)∈ℋ,ve≔Φ(pe)∈ℋ,u_a (t_a) , u_a ( t_a) , v_e (p_e) , and collect them into linear maps T∗=[u1⋯uk],T¯=[u¯1⋯u¯k],P=[v1⋯vm].T^*= bmatrixu_1&·s&u_k bmatrix, T= bmatrix u_1&·s& u_k bmatrix, P= bmatrixv_1&·s&v_m bmatrix. By the ground-truth mixture identity, feasibility, and linearity of Φ , P=T¯Π=T∗Π∗.P= T =T^* ^*. By 3(i), rank(GΦ(t1,…,tk))=k,rank\! (G_ (t_1,…,t_k) )=k, so the Gram matrix GΦ(t1,…,tk)G_ (t_1,…,t_k) is positive definite. Hence u1,…,uku_1,…,u_k are linearly independent, and therefore dimspanu1,…,uk=k. \u_1,…,u_k\=k. Moreover, since Π∗ ^* is sufficiently scattered, the cone K≔cone(Π∗)K ( ^*) contains kC_k, which is full-dimensional in ℝkR^k. Hence K is full-dimensional, so rank(Π∗)=krank( ^*)=k. Therefore the columns of Π∗ ^* span ℝkR^k, and thus spanv1,…,vm=T∗(ℝk)=spanu1,…,uk.span\v_1,…,v_m\=T^*(R^k)=span\u_1,…,u_k\. In particular, dimspanv1,…,vm=k. \v_1,…,v_m\=k. On the other hand, since P=T¯ΠP= T , we have spanv1,…,vm⊆spanu¯1,…,u¯k.span\v_1,…,v_m\ \ u_1,…, u_k\. The left-hand side has dimension k, while the right-hand side is spanned by at most k vectors, so in fact spanu¯1,…,u¯k=spanu1,…,uk.span\ u_1,…, u_k\=span\u_1,…,u_k\. Therefore there exists a unique invertible matrix S∈GL(k)S (k) such that T¯=T∗S. T=T^*S. Substituting into P=T¯Π=T∗Π∗P= T =T^* ^* gives T∗SΠ=T∗Π∗.T^*S =T^* ^*. Since the columns of T∗T^* are linearly independent, T∗T^* is injective on ℝkR^k, and hence SΠ=Π∗,soΠ=S−1Π∗.S = ^*, =S^-1 ^*. Let A≔S−1.A S^-1. Then AΠ∗=Π≥0.A ^*= ≥ 0. Next, because Φ is linear and injective, the identity T¯=T∗S T=T^*S implies that for each a∈[k]a∈[k], t¯a=∑b=1kSbatb t_a= _b=1^kS_ba\,t_b as finite signed measures in ℳ()M(O). Since both t¯a t_a and each tbt_b are probability measures, taking total mass yields 1=t¯a()=∑b=1kSbatb()=∑b=1kSba.1= t_a(O)= _b=1^kS_ba\,t_b(O)= _b=1^kS_ba. Thus every column of S sums to one, i.e. ⊤S=⊤. 1 S= 1 . Equivalently, A⊤=.A 1= 1. We may therefore apply Lemma A.2 to conclude that |det(A)|≤1,hence|det(S)|≥1,| (A)|≤ 1, | (S)|≥ 1, with equality if and only if S is a permutation matrix. Finally, the embedded Gram matrix transforms as GΦ(t¯1,…,t¯k)=S⊤GΦ(t1,…,tk)S,G_ ( t_1,…, t_k)=S G_ (t_1,…,t_k)S, and therefore det(GΦ(t¯1,…,t¯k))=det(GΦ(t1,…,tk))det(S)2. (G_ ( t_1,…, t_k) )= (G_ (t_1,…,t_k) )\, (S)^2. Since |det(S)|≥1| (S)|≥ 1, we obtain det(GΦ(t¯1,…,t¯k))≥det(GΦ(t1,…,tk)). (G_ ( t_1,…, t_k) )≥ (G_ (t_1,…,t_k) ). The ground-truth factorization (t1,…,tk,Π∗)(t_1,…,t_k, ^*) is feasible, so it achieves the value det(GΦ(t1,…,tk)). (G_ (t_1,…,t_k) ). Hence every optimizer must satisfy |det(S)|=1| (S)|=1. By Lemma A.2, this happens if and only if S is a permutation matrix. Therefore there exists a permutation matrix Σ such that T¯=T∗Σ,Π=Σ⊤Π∗. T=T^* , = ^*. Because Φ is injective, T¯=T∗Σ T=T^* implies t¯a=tΣ(a),a∈[k]. t_a=t_ (a), a∈[k]. Restoring the dependence on o gives exactly the claimed conclusion: t¯a=tΣ(a)o,a∈[k],andΠo=Σo⊤Πo∗. t_a=t^o_ (a), a∈[k], _o= _o _o^*. ∎ A.4 Proof of Theorem 4.3 Fix o0∈o_0 , and let Σ0≔Σ(o0) _0 (o_0). Write σ0 _0 for the corresponding permutation of [k][k], so that ϕa(o0)=ϕσ0(a)∗(o0)for all a∈[k]. _a(o_0)= _ _0(a)^*(o_0) all a∈[k]. Because the true embedded latent transitions at o0o_0 are pairwise distinct, the quantity δ0≔mini≠j‖ϕi∗(o0)−ϕj∗(o0)‖ℋ _0 _i≠ j \| _i^*(o_0)- _j^*(o_0) \|_H is strictly positive. By continuity of o↦ϕa(o)o _a(o) and o↦ϕa∗(o)o _a^*(o), there exists a neighborhood U of o0o_0 such that for all o∈Uo∈ U and all a∈[k]a∈[k], ‖ϕa(o)−ϕa(o0)‖ℋ<δ0/4,‖ϕa∗(o)−ϕa∗(o0)‖ℋ<δ0/4.\| _a(o)- _a(o_0)\|_H< _0/4, \| _a^*(o)- _a^*(o_0)\|_H< _0/4. Fix o∈Uo∈ U. For each a∈[k]a∈[k], we then have ‖ϕa(o)−ϕσ0(a)∗(o)‖ℋ≤‖ϕa(o)−ϕa(o0)‖ℋ+‖ϕσ0(a)∗(o0)−ϕσ0(a)∗(o)‖ℋ<δ0/2.\| _a(o)- _ _0(a)^*(o)\|_H≤\| _a(o)- _a(o_0)\|_H+\| _ _0(a)^*(o_0)- _ _0(a)^*(o)\|_H< _0/2. On the other hand, for any b≠σ0(a)b≠ _0(a), ‖ϕa(o)−ϕb∗(o)‖ℋ \| _a(o)- _b^*(o)\|_H ≥‖ϕσ0(a)∗(o0)−ϕb∗(o0)‖ℋ−‖ϕa(o)−ϕa(o0)‖ℋ−‖ϕb∗(o)−ϕb∗(o0)‖ℋ ≥\| _ _0(a)^*(o_0)- _b^*(o_0)\|_H-\| _a(o)- _a(o_0)\|_H-\| _b^*(o)- _b^*(o_0)\|_H >δ0−δ0/4−δ0/4=δ0/2. > _0- _0/4- _0/4= _0/2. Thus, for every a∈[k]a∈[k], the vector ϕa(o) _a(o) is strictly closer than δ0/2 _0/2 to ϕσ0(a)∗(o) _ _0(a)^*(o) and strictly farther than δ0/2 _0/2 from every other true component ϕb∗(o) _b^*(o). Therefore the only permutation relating T(o)T(o) and T∗(o)T^*(o) is Σ0 _0. Hence Σ(o)=Σ0for all o∈U. (o)= _0 all o∈ U. So Σ(⋅) (·) is locally constant. Since Σ(o) (o) takes values in the finite discrete set of k×k× k permutation matrices, any locally constant map is constant on each connected component of O. Because O is connected, Σ(o) (o) must be constant on all of O. A.5 Proof of Corollary 4.1 Fix any o∈o . By 5 (i)–(i), Theorem 4.2 applies at that observation. Hence there exists a permutation matrix Σ(o) (o) such that p(⋅∣o,a)=p∗(⋅∣o,Σ(o)(a)),Π(o)=Σ(o)⊤Π∗(o).p(· o,a)=p^*(· o, (o)(a)), (o)= (o) ^*(o). Because Φ is injective, this is equivalent to T(o)=T∗(o)Σ(o),Π(o)=Σ(o)⊤Π∗(o),T(o)=T^*(o) (o), (o)= (o) ^*(o), where T(o)=[Φ(p(⋅∣o,1))⋯Φ(p(⋅∣o,k))],T∗(o)=[Φ(p∗(⋅∣o,1))⋯Φ(p∗(⋅∣o,k))].T(o)= bmatrix (p(· o,1))&·s& (p(· o,k)) bmatrix, T^*(o)= bmatrix (p^*(· o,1))&·s& (p^*(· o,k)) bmatrix. By 5 (iv) and the continuity assumed for the candidate family, the maps o↦T(o)o T(o) and o↦T∗(o)o T^*(o) are continuous. Moreover, 5 (i) implies that the embedded true latent transitions are linearly independent at every o, and in particular pairwise distinct. Therefore the hypotheses of Theorem 4.3 are satisfied, so there exists a single permutation matrix Σ such that p(⋅∣o,a)=p∗(⋅∣o,Σ(a)),Π(o)=Σ⊤Π∗(o)for all o∈.p(· o,a)=p^*(· o, (a)), (o)= ^*(o) all o . Finally, by 5 (v), the permutation is known on the anchor subset S. Since the same permutation Σ applies globally, it is uniquely determined there and hence uniquely determined everywhere. If the anchor identifies the true labeling, then Σ=Ik =I_k, yielding p(⋅∣o,a)=p∗(⋅∣o,a),Π(o)=Π∗(o)for all o∈,a∈[k].p(· o,a)=p^*(· o,a), (o)= ^*(o) all o ,\ a∈[k]. This proves the claim.