Paper deep dive
IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models
Hamed Rahimi, Clemence Grislain, Adrien Jacquet Cretides, Olivier Sigaud, Mohamed Chetouani
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 6/21/2026, 7:47:26 AM
Summary
IntentVLM is a novel two-stage video-language framework designed for open-vocabulary human intention recognition in social robotics. Inspired by forward-inverse modeling from cognitive science, the model decomposes intention understanding into two steps: first, generating a set of plausible goal candidates from multimodal inputs (video and text), and second, performing structured inference to select the most likely intention. This hierarchical approach aims to reduce hallucinations and improve reasoning in unconstrained, real-world environments. The framework utilizes two frozen video-language models augmented with trained LoRA adapters and achieves state-of-the-art performance on the IntentQA and Inst-IT Bench datasets.
Entities (6)
Relation Signals (4)
IntentVLM â evaluatedon â IntentQA
confidence 100% · Evaluated on the IntentQA and Inst-IT Bench datasets
IntentVLM â isinspiredby â Forward-Inverse Modeling
confidence 100% · The approach is inspired by forward-inverse modeling in cognitive science by decomposing intention understanding into goal candidate generation followed by structured inference through selection
IntentVLM â uses â LoRA
confidence 100% · implemented with two frozen videoâlanguage modules augmented by trained LoRA adapters.
IntentVLM â developedat â Sorbonne University
confidence 90% · Institute of Intelligent Systems and Robotics (ISIR), Sorbonne University Paris, France
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Improving the effectiveness of human-robot interaction requires social robots to accurately infer human goals through robust intention understanding. This challenge is particularly critical in multimodal settings, where agents must integrate heterogeneous signals including text, visual cues to form a coherent interpretation of user intent. This paper presents IntentVLM, a novel two-stage video-language framework designed for open-vocabulary human intention recognition. The approach is inspired by forward-inverse modeling in cognitive science by decomposing intention understanding into goal candidate generation followed by structured inference through selection, effectively reducing hallucinations in latent reasoning. Evaluated on the IntentQA and Inst-IT Bench datasets, IntentVLM achieves state-of-the-art results with up to 80% accuracy, notably surpassing the baseline performance by 30% and matches human performance. Our findings demonstrate that this structured reasoning approach enhances open-vocabulary intention understanding without catastrophic forgetting, offering a robust foundation for human-centered robotics.
Tags
Links
- Source: https://arxiv.org/abs/2604.24002v1
- Canonical: https://arxiv.org/abs/2604.24002v1
Trouble viewing inline? Open PDF directly â
Full Text
55,568 characters extracted from source content.
Expand or collapse full text
by IntentVLM: Open-Vocabulary Intention Recognition through ForwardâInverse Modeling with Video-Language Models Hamed Rahimi, ClĂ©mence Grislain, Adrien Jacquet CrĂ©tides, Olivier Sigaud, Mohamed Chetouani Institute of Intelligent Systems and Robotics (ISIR), Sorbonne University Paris, France firstname.lastname@isir.upmc.fr (April 2026) Abstract. Improving the effectiveness of humanârobot interaction requires social robots to accurately infer human goals through robust intention understanding. This challenge is particularly critical in multimodal settings, where agents must integrate heterogeneous signals including text, visual cues to form a coherent interpretation of user intent. This paper presents IntentVLM, a novel two-stage video-language framework designed for open-vocabulary human intention recognition. The approach is inspired by forward-inverse modeling in cognitive science by decomposing intention understanding into goal candidate generation followed by structured inference through selection, effectively reducing hallucinations in latent reasoning. Evaluated on the IntentQA and Inst-IT Bench datasets, IntentVLM achieves state-of-the-art results with up to 80% accuracy, notably surpassing the baseline performance by 30% and matches human performance. Our findings demonstrate that this structured reasoning approach enhances open-vocabulary intention understanding without catastrophic forgetting, offering a robust foundation for human-centered robotics. Intention Recognition, Video-Language Models, Parameter Efficient Finetuning â journalyear: 2026â copyright: câ conference: Proceedings of the 28th International Conference on Multimodal Interaction; October 5â9, 2026; Napoli, Italyâ booktitle: Proceedings of the 28th International Conference on Multimodal Interaction (ICMI â26), October 5â9, 2026, Napoli, Italyâ copyright: noneâ ccs: Computing methodologies Artificial intelligenceâ ccs: Human-centered computing Human computer interaction (HCI)â ccs: Computing methodologies Machine learning approaches Figure 1. IntentVLM for open-vocabulary human intention recognition from video streams. Given a continuous video stream of a human in a kitchen environment, IntentVLM integrates multimodal visual cues â scene context, object interactions, and body movements â to infer free-form user intentions without relying on a predefined action vocabulary. The model reasons over temporal video evidence to produce natural language intention descriptions (âMake coffeeâ, âHeat water for teaâ), demonstrating the capacity of video-language understanding to ground open-vocabulary human goals in real-world robotic settings. Code and Data 1. Introduction As robots increasingly transition from controlled industrial environments into everyday proactive robot human-centered spaces (van Den Broek et al., 2024), their ability to understand and respond appropriately to human behavior becomes critically important(MalĂ©cot et al., 2026). In such settings, effective interaction requires more than simply perceiving observable actions, meaning robots must also reason about the underlying human intentions that drive those actions. Human intention refers to a mental state representing a goal-directed commitment to performing a particular action (BrĂŒne et al., 2007; Chetouani, 2026). Within the framework of mental state attribution, intentions function as internal states that guide and motivate behavior, enabling an observer to interpret why an agent behaves in a particular way. For example, as shown in Figure 1, when a person enters a reception area and begins scanning the room (e.g. looking at people, objects, or walls) their behavior cannot be misinterpreted as casual curiosity, but in reality, this pattern more commonly signals an underlying intention to seek guidance or assistance (Ziemke, 2020). Recognizing such subtle behavioral cues is essential to accurately infer user intent and respond appropriately. Figure 2. Closed-vocabulary solutions vs our approach. Conventional approaches formulate intention recognition as closed-set classification over a predefined label space given visual inputs. Inspired by forwardâinverse modeling in cognitive science, our method removes this requirement by decomposing the task into (i) goal candidate generation and (i) structured selection via a videoâlanguage model. This two-stage formulation reduces hallucinations in latent reasoning and supports open-vocabulary intention prediction. While intention is often discussed in the context of human cognition, it is not exclusive to humans. Animals exhibit goal-directed behavior, and autonomous robotic systems act according to internal models and task-oriented objectives (Thellman et al., 2022). In this broader view, an agentâs behavior can be understood through the interplay of beliefs (probabilistic representations of the world), goals (desired states to be achieved), intentions (commitments to particular courses of action), and policies (action-selection mechanisms) (Chetouani, 2026). Consequently, intention recognition can be defined as the process of inferring an agentâs latent intentions (i.e., its committed plans or policies) by reasoning over observable actions in relation to underlying beliefs, goals, and environmental constraints. Rather than focusing solely on what an agent does, intention recognition seeks to explain why those actions are performed and to predict future behavior accordingly (Hoffman et al., 2024). Despite its importance, intention recognition remains a challenging problem (Jain and Argall, 2019). The primary difficulty arises from the fact that intentions are inherently unobservable. Unlike physical actions, which can be directly perceived through sensory input, intentions exist only as latent mental states that must be inferred indirectly (Hoffman et al., 2024). Observers therefore infer an agentâs intention, which is its underlying goal-directed commitment, by integrating behavioral patterns, contextual information, prior knowledge, and communicative cues (CrĂ©tides et al., 2026). Furthermore, intention expression is inherently multimodal (Trick et al., 2019). Humans convey intent through interactions with the environment with a combination of visual and linguistic signals. Another challenge stems from the imperfect relationship between actions and intentions (Nguyen et al., 2011). Not every intention culminates in an observable action, nor does every action faithfully reveal the agentâs underlying intention. A goal denotes the desired outcome an individual seeks to achieve, whereas intention captures the specific commitment or plan to act toward that outcome. In practice, these constructs may diverge: an agent may intend to perform a beneficial action yet fail due to external constraints, or may execute an apparently helpful behavior while pursuing a self-serving objective (Grislain et al., 2025). Such misalignment between goals, intentions, and observable actions introduces substantial ambiguity in behavior interpretation, thereby making robust intention inference a central challenge for robotic systems. A large body of research has addressed intention recognition from multiple perspectives (Hoffman et al., 2024), including Bayesian inverse planning (Buyukgoz et al., 2022), probabilistic reasoning (Van-Horenbeke and Peer, 2021), plan recognition (Kautz and Allen, 1986), and learning-based approaches (Sukthankar et al., 2014). However, as shown in Figure 2 many existing methods simplify the problem by restricting possible intentions to a predefined set, often represented by a limited vocabulary of intention labels (Zhao et al., 2025). While such formulations simplify learning and evaluation, they significantly restrict the expressiveness of intention modeling. Human intentions frequently involve nuanced psychological states, social goals, or compound motivations that cannot be easily represented by a single discrete label (Buyukgoz et al., 2022). Additionally, a large portion of prior work relies primarily on static image-based perception, which limits the systemâs ability to capture temporal dynamics and contextual evolution present in real-world scenes. As a result, integrating intention recognition into complex, temporally evolving visual scenes remains an open challenge. To address these limitations, as shown in Figure 3, we introduce IntentVLM, a video foundation model (Madan et al., 2024) designed for open-ended human intention understanding. We formulate intention recognition as a cognitive modeling problem grounded in forwardâinverse reasoning (Ho and Griffiths, 2022a), and instantiate it as a videoâlanguage reasoning task using two visionâlanguage models (VLMs) (Zhang et al., 2024). Given a video of a scene and a textual query about the intention of an observed agent, the system outputs a textual description of the inferred intention. Our framework comprises two complementary modules. The first generates a diverse set of candidate intentions conditioned on the video context, while the second evaluates and ranks these candidates via multimodal reasoning to select the most plausible one. This two-stage design enables broader exploration of the intention space while reducing hallucinations and improving reliability. Extensive experiments demonstrate that the proposed approach provides more accurate and contextually grounded intention predictions compared to baseline video foundation models. Moreover, the modular design enables improved computational efficiency and reduced model complexity while maintaining strong reasoning capability. These results establish structured intention recognition as a key component for enabling video-language foundation models to support more socially aware and cognitively capable robotic systems. Figure 3. Architecture of IntentVLM. We cast the model as a cognitive framework based on forwardâinverse reasoning, implemented with two frozen videoâlanguage modules augmented by trained LoRA adapters. The first module generates a diverse set of candidate intentions for each videoâquery pair, while the second acts as an expert that evaluates and selects the most likely intention. The model is trained on egocentric video data, where a robot seeks to infer human intentions from its embodied viewpoint. The LoRA adapters are optimized using a cross-entropy loss computed between the ground-truth labels (intention candidates for the first module and the true human intention for the second module) and the tokens generated by the video-language model. 2. Related Work Intention recognition, closely related to goal recognition (Lesh and Etzioni, 1995) and plan recognition (Kautz and Allen, 1986), refers to the process of inferring an agentâs goals and plans from observed behavior. Intention, goal, and plan are closely related notions, which helps explain why similar methodological frameworks are often employed to infer them. However, this similarity is not only conceptual but also practical: all three tasks rely on interpreting observable actions to reconstruct latent structures, often under uncertainty and partial observability, leading to shared modeling assumptions and inference techniques. Despite these commonalities, intention recognition can be distinguished as a broader process, as it aims to infer an actorâs underlying mental states and high-level objectives as they are manifested through actions (Bratman, 1989). In this sense, it subsumes aspects of both goal and plan recognition while extending beyond them. In the literature, developed intention recognition systems are broadly categorized into probabilistic and logic-based approaches (Van-Horenbeke and Peer, 2021). A foundational framework for formalizing this problem is Bayesian Inverse Planning, which frames intention recognition as the inversion of an agentâs decision-making process (Charniak and Goldman, 1991). By formulating the task as a Markov Decision Process, probabilistic methods (Baker et al., 2009b; RamĂrez and Geffner, 2009, 2010) define intent as a latent variable that maximizes the likelihood of observed actions. This approach is particularly effective at managing real-world uncertainty and the inherent ânon-rationalityâ of human behavior. However, these models often face scalability challenges due to state-space explosion. To mitigate this, Schrempf et al. (Schrempf et al., 2007) utilize expert knowledge to derive reduced state spaces for hidden variables within Bayesian networks. In contrast, logic-based methods rely on plan representation languages and domain-independent rules to deduce intent through formal reasoning (Sukthankar et al., 2014; Masters and Sardina, 2019). While these approaches offer high expressivity and transparency, they often struggle to model the stochastic nature of human actions or sub-optimal decision-making (Dreyfus, 2007). To bridge this gap, hybrid frameworks have been developed to combine the deductive strength of logic with the robustness of Bayesian inference (Buyukgoz et al., 2022), aiming to provide systems that are both interpretable and capable of handling environmental noise. Prior work in cognitive science has framed intention recognition as an instance of inverse modeling, tightly coupled with forward predictive processes. Early accounts emphasized that predictive (forward) models of othersâ behavior rely on prior inference over communicative or action intentions, rather than purely stimulus-driven prediction (de Ruiter and Cummins, 2013). More recent computational approaches formalize this relationship using Bayesian inverse planning, where observers infer latent goals by assuming approximately rational action, and combine this with forward simulation to predict future behavior (Baker et al., 2009a; Qian et al., 2021). This forwardâinverse loop has been further developed in frameworks such as inverse reinforcement learning, which infer underlying reward structures or shifting intentions from observed trajectories (Ng and Russell, 2000; Hadfield-Menell et al., 2016). Integrative perspectives (Ho and Griffiths, 2022b) unify these approaches under a broader view of Theory of Mind as inverse inference over decision-making processes, supported by forward generative models for prediction (Abrini et al., 2025). The emergence of Foundation Models has shifted the paradigm of embodied AI from narrow task-specific solvers to general-purpose reasoning agents (Huang et al., 2022a, b; Khan and Waheed, 2025). Unlike classical models that require hard-coded rules, Large Language Models (LLMs) possess commonsense priors developed through massive-scale pre-training (Brown et al., 2020). This allows for âzero-shotâ inference of high-level objectives, where a model can hypothesize an agentâs intent by grounding observed actions in its broad world knowledge (Wei et al., 2022). Vision-Language Models (VLMs) extend this reasoning capability to the physical world by aligning visual perceptions with linguistic semantics (Li et al., 2025; Rahimi et al., 2025a, c). Early alignment efforts, such as contrastive frameworks like CLIP (Radford et al., 2021) introduced open-vocabulary recognition, enabling systems to identify novel objects and actions without predefined categories (Gao et al., 2024). This is critical for intention recognition in unconstrained environments, where the set of possible human goals is theoretically infinite. Modern modular architectures, such as Qwen3-VL (Bai et al., 2025), further refine this by using lightweight adapters to bridge frozen visual encoders with LLM reasoning cores (Rahimi et al., 2025b, d). This evolution ensures that intention recognition is no longer confined to predefined state spaces but can instead leverage the world knowledge of foundation models to interpret human behavior in unconstrained settings. Prior work studied LLMs for intent understanding in social interactions (Sap et al., 2022) and robotic QA (Chiu et al., 2021), as well as RL-based goal inference in dynamic and collaborative settings (Nageris et al., 2024; Lin et al., 2025). More recent approaches use LLMs to infer goals from observed actions or dialogue (Ying et al., 2024; Zhang et al., 2025b; Wan et al., 2025). While early multimodal methods exist (Z et al., 2021), VLMs now enable grounding language into actions and tracking long-term intent; e.g., LIT (Huang et al., 2024) models behavior from video, and OV-MER (Lian et al., 2025) adds open-vocabulary emotion recognition. Building on these developments, our work addresses a key limitation in existing approaches: the lack of structured reasoning mechanisms for dynamic multimodal, open-vocabulary intention understanding. While recent LLM- and VLM-based methods demonstrate strong zero-shot capabilities, they often rely on implicit, single-pass inference, which can lead to hallucinations and inconsistent goal predictions in complex, real-world scenarios. Our design enables more robust integration of heterogeneous signals such as video and language, while preserving the flexibility of foundation models in open-world settings. By combining the expressivity of VLMs with an explicit reasoning structure, our approach bridges the gap between classical probabilistic inference and modern foundation model capabilities, yielding improved accuracy and human-level performance on challenging benchmarks. 3. Method 3.1. Problem Definition Intentions are latent components of humansâ cognitive state that govern goal-directed behavior. Although intentions are not directly observable, they can be inferred indirectly through the analysis of observed actions and multimodal behavioral cues. We therefore formulate intention recognition as the problem of inferring humanâs underlying goal and intention conditioned on partial observations of their behavior. Consider an observer monitoring a human interacting with an environment over time. The observer perceives a sequence of state-action pairs forming an observation trajectory: (1) O0:T=âšS0,A0,S1,A1,âŠ,ST,ATâ©,O_0:T= S_0,A_0,~S_1,A_1,~âŠ,~S_T,A_T , where StâS_t denotes the observed environmental state at time t, AtâA_t represents the low-level action performed by the agent. The sequence captures the observable evolution of the interaction between humans and their environment. However, in realistic scenarios the observer does not have direct access to these structured variables. Instead, the observer receives multimodal sensory inputs such as visual observations and linguistic queries. These observations provide only partial information about the underlying state, action, and goal structure governing the agentâs behavior. We model the latent intention guiding the behavior of humans as a hidden variable ItââI_t representing the internal objective of the agent at time t. While goal and intention are conceptually distinct in the cognitive literature, in this work we assume that the observed agent is neither deceptive nor pursuing unattainable goals. Under these constraints, the agentâs observable behavior reliably reflects both their goal and their intention, allowing us to treat the two concepts as interchangeable for the purposes of this work. In our setting, the observing robot interacts with two primary modalities: an egocentric visual stream obtained from onboard cameras and a textual query that may originate from a human user or from the robotâs internal reasoning module. Let VOV_O denote the observed video sequence and QOQ_O denote the textual query describing the intention-related question. These modalities offer uncertain evidence of the humanâs behavior and surrounding environment, yet neither the agentâs goal GtG_t nor latent intention ItI_t is directly observable. The robot must therefore reason under partial observability. We hypothesize that explicitly modeling an intermediate goal inference step improves intention understanding, drawing inspiration from forward-inverse modeling frameworks in cognitive science, where the brain is thought to first simulate plausible future states before reasoning backward to infer the underlying intent of an action (Ho and Griffiths, 2022b). Rather than predicting intention directly from multimodal observations, we first infer a set of plausible goal candidates and subsequently reason over them to infer the most likely intention. This hierarchical formulation encourages the model to interpret observations in terms of goal-oriented behavior before making a higher-level cognitive inference about intention â mirroring the forward modelâs role in anticipating outcomes and the inverse modelâs role in attributing intent. Furthermore, grounding this process in a a video-language model naturally extends the framework to an open-vocabulary setting, enabling the system to express and reason over intentions beyond a fixed label space, capturing the context-dependent diversity of human goals in unconstrained environments. Such a decomposition allows the system to leverage richer contextual cues and reduces ambiguity in the final prediction. 3.2. IntentVLM To investigate this hypothesis, we propose IntentVLM, a video-language model for multimodal intention reasoning. The framework comprises two sequential componentsâa goal candidate generator and an intention inference moduleâthat operate in two stages: Stage 1: Proposal The first module takes as input the observed video VOV_O and the textual query QOQ_O, and produces a set of K plausible goal candidates, that explain the observed behavior, in form of multiple choices. Intuitively, these goals correspond to intermediate hypotheses about the humanâs underlying intent in the scene. We model this component as a conditional generative model built upon a video-language model parameterized by ÏÏ. Formally, the model generates a set of candidate goals conditioned on the observed modalities: (2) t=FÏâ(VO,QO),G_t=F_Ï(V_O,Q_O), where t=G1,âŠ,GKG_t=\G_1,âŠ,G_K\ denotes the set of generated candidate goals, which may belong to either a fixed vocabulary or an open vocabulary. Here, FÏF_Ï represents the parameterized video-language model that produces candidate goals based on the visual observations and the textual query. Stage 2: Selection The second module performs intention inference by evaluating candidate goals and selecting the one that best explains the observed behavior. As mentioned before, we treat this selected goal as the inferred human intention, leveraging the close relationship between goals and intentions in goal-directed behavior: intentions reflect a commitment to achieve a desired outcome, while goals specify that outcome. Under this view, identifying the goal that most plausibly accounts for the observed actions provides a practical proxy for intention inference. Concretely, the model ranks candidate goals based on their consistency with the visual evidence and contextual information, and selects the one that maximizes explanatory alignment with the agentâs behavior. Formally, let t=Gt1,Gt2,âŠ,GtiG_t=\G_t^1,G_t^2,âŠ,G_t^i\ denote the candidate goal set generated by the first module. In practice, we retain only the top-5 candidates from the first module, ensuring that the second-stage scoring remains tractable. The second module evaluates each candidate conditioned on the observed modalities and assigns them a likelihood score. We parameterize this scoring function using a model with parameters Ξ: (3) PΞâ(GtkâŁVO,QO,t),P_Ξ(G_t^k V_O,Q_O,G_t), where GtkG_t^k represents a candidate goal from the generated set. The inferred intention is obtained by selecting the goal with the highest posterior probability: (4) It=argâĄmaxGtkâtâĄPΞâ(GtkâŁVO,QO,t).I_t= _G_t^k _tP_Ξ(G_t^k V_O,Q_O,G_t). The selected goal GtâG_t^* is therefore treated as the estimated intention ItI_t of the agent. This formulation allows the model to explicitly reason over multiple plausible interpretations before committing to the most consistent explanation of the observed behavior. Figure 4. IntentQA Benchmark (Li et al., 2023). Illustrative examples of the four QA types in our training dataset. CW: a man points at a screen to guide a childâs attention. CH: a pointing gesture expresses the intent to drink. TN: a boyâs pointing causes the baby to look in that direction. TP: a girlâs pointing is triggered by the babyâs action. The red box indicates the correct answer. Figure 5. Inst-IT Benchmark (Peng et al., 2024). Each sample consists of temporally grounded frame-level annotations with instance IDs, a coherent video-level description, and instance-focused QA pairs. The annotations capture fine-grained attributes, interactions, and temporal changes of instances across frames, enabling explicit instance-level reasoning. 4. Experiments To evaluate the effectiveness of the proposed framework, we train two LoRA adapters corresponding to the two stages of IntentVLM: (i) goal candidate generation and (i) intention inference via goal selection. Our experimental study is designed to answer the following key questions: (1) How do video-language models perform on intention recognition compared to existing baseline methods? (2) Does an open-vocabulary formulation combined with a two-stage (proposal and selection) framework improve performance? (3) Does training for intention reasoning improve generalization, or does it negatively affect visual understanding capabilities? (4) How does the scale of the underlying video-language model impact performance? To systematically investigate these questions, we design an experimental framework comprising two benchmarks. The first benchmark addresses the initial two research questions, focusing on (i) plain intention recognition and (i) open-vocabulary comparison. The second benchmark evaluates the modelâs susceptibility to catastrophic forgetting, as well as its scalability, particularly in terms of the performance-to-size trade-off. Model Text Rep. CW CH TP&TN Total Val. Test Val. Test Val. Test Val. Test EVQAâ GloVe 25.99 25.92 37.43 34.54 28.00 25.52 28.38 27.27 CoMemâ GloVe 31.56 30.00 35.63 28.69 28.57 28.95 31.46 29.52 HGAâ GloVe 29.45 32.00 35.03 30.64 29.71 31.05 30.43 31.54 HMEâ GloVe 30.97 34.40 35.33 34.26 34.29 29.14 32.53 33.08 HQGAâ GloVe 32.49 33.20 38.32 34.26 34.48 36.57 33.95 34.21 CoMemâ BERT 46.75 47.68 57.49 54.87 41.71 39.05 47.21 46.77 HGAâ BERT 43.54 44.88 56.89 50.97 42.48 39.62 45.45 44.61 HMEâ BERT 46.50 46.08 51.20 54.32 44.76 40.76 46.82 46.16 HQGAâ BERT 45.91 48.24 57.78 54.32 44.76 41.71 47.55 47.66 VGTâ BERT 50.46 51.44 55.99 55.99 48.19 47.62 50.78 51.27 Blind GPTâ â â 52.16 â 61.28 â 43.43 â 51.55 VideoQAâ w/o GPT BERT 52.57 55.28 57.47 61.56 46.10 47.81 51.71 54.50 VideoQAâ BERT â 58.40 â 65.46 â 50.48 â 57.64 IntentVLM (ours) â 85.30 84.10 83.80 88.60 75.30 83.95 79.93 85.15 Humanâ â â 77.76 â 80.22 â 79.05 â 78.49 Table 1. Comparison of different models and Total accuracy (%). â Values are reported by (Li et al., 2023). 4.1. Datasets We evaluate our model on two multimodal benchmarks. The first focuses on intention recognition and understanding. The second assesses whether the model avoids catastrophic forgetting while preserving global scene understanding in complex tasks. In particular, we examine its performance on instance-level understanding and reasoning. Instance detection identifies what objects are present and where, while intention detection infers the underlying goal or purpose behind an observation. IntentQA. The first dataset, IntentQA (Li et al., 2023), is designed specifically for intention recognition in video contexts. Each sample in the dataset consists of a video, an intention-related question, a set of candidate answers, and a ground-truth label. Formally, each data point can be represented as: (5) x=(V,Q,,aâ),x=(V,Q,A,a^*), where V denotes the input video, Q is the intention-related query, =a1,a2,âŠ,a5A=\a_1,a_2,âŠ,a_5\ represents a set of five candidate answers, and aââa^* is the correct answer. The dataset is divided into four categories based on the type of reasoning required: Causal What (CW): understanding what intention explains an observed action, Causal How (CH): reasoning about how an intention leads to a specific outcome, Temporal Next (TN): predicting future actions based on inferred intention, Temporal Past (TP): inferring past intentions from observed outcomes. An example of this dataset is provided in Figure 4. Inst-IT Bench. The second dataset, Inst-IT Bench (Peng et al., 2024), is a fine-grained multimodal benchmark designed to evaluate instance-level comprehension in both images and videos. It provides a more detailed assessment of a modelâs ability to reason about specific entities and their interactions within a scene. Formally, each sample in Inst-IT Bench can be represented as: (6) x=(M,Q,,aâ),x=(M,Q,A,a^*), where M denotes the input modality (either an image or a video), Q is the query, A is the set of candidate answers (or a free-form answer space in the open-ended setting), and aâa^* is the ground-truth answer. Inst-IT Bench consists of two splits: Image Split containing 1,036 question-answer pairs over 338 images, and Video Split containing 1,001 question-answer pairs over 206 videos. Each question-answer pair is provided in both multiple-choice and open-ended formats, enabling comprehensive evaluation of both constrained reasoning and generative capabilities. An example of this dataset is provided in Figure 7. As the main goal of this paper is intention recognition, we train our models on the train set IntentQA dataset and evaluate the trained model on the test and Validation sets of IntentQA plus on the test set of Inst-IT bench. Model Setting ROUGE-1 ROUGE-L CosSim BERTScore-F1 Val Test Val Test Val Test Val Test Qwen3-VL 2B Zero-shot 17.18 15.60 16.88 15.30 33.35 31.13 87.78 87.33 Qwen3-VL 2B Finetuned 13.05 14.10 12.35 13.30 35.15 35.53 86.00 86.18 Qwen3-VL 4B Zero-shot 19.71 19.75 18.95 19.00 41.07 41.04 88.68 88.55 Qwen3-VL 4B Finetuned 12.89 13.20 12.63 12.98 35.85 35.94 85.82 85.98 IntentVLM (Ours) 2-step (ft) 21.05 19.18 20.48 18.91 35.75 34.67 87.75 87.20 Table 2. Comparison of intention understanding performance on the IntentQA dataset, evaluated with and without answer options. All values are reported as percentages (%) and averaged across CW, CH, TP, and TN metrics. Figure 6. Ablation study analyzing the impact of VLM scale (2B vs. 4B) and training regime (zero-shot vs. fine-tuned) on performance. 4.2. Training Procedure For training, we adopt Qwen3-VL (Bai et al., 2025) as the backbone video-language model and fine-tune it separately for each stage of the proposed framework. To ensure computational efficiency, we employ parameter-efficient fine-tuning (PEFT) (Zhang et al., 2025a) using Low-Rank Adaptation (LoRA) (Hu et al., 2022), allowing the model to adapt to the task with a minimal number of trainable parameters. Goal Candidate Proposal. In the first stage, the model is trained to generate a set of plausible goal (intention) candidates given the multimodal input. The input consists of the video V, the query Q, and an instruction U that prompts the model to generate multiple candidate options. The supervision signal is provided by the candidate options available in the IntentQA dataset. Formally, this stage models the conditional distribution over goal candidates: GâŒFÏâ(V,Q,U),G F_Ï(V,Q,U), where G=g1,g2,âŠ,gKG=\g_1,g_2,âŠ,g_K\ denotes the set of generated candidate goals. Here, FÏF_Ï is a videoâlanguage model parameterized by ÏÏ (implemented via LoRA adapters), V is the input video, Q is the textual query, and U is the instruction prompt. Intention Selection. In the second stage, we fine-tune another instance of the Qwen3-VL backbone to perform intention inference by selecting the most plausible option from the candidate set. The model takes as input the video V, the query Q, the instruction U, and the set of candidate options G, and outputs the selected intention. Formally, this stage is defined as: PΞâ(iâŁU,V,Q,G),where âiâOP_Ξ(i U,V,Q,G), iâ O, where i denotes the selected intention (i.e., the most likely goal) from the candidate set O. (a) (b) Figure 7. Performance on the Inst-IT benchmark. (a) shows that our model remains highly competitive with baseline methods, indicating that training does not degrade its scene understanding on this complex instance-level task. (b) demonstrates the modelâs superior sizeâperformance trade-off, with a 4B-parameter model achieving performance comparable to models exceeding 7B parameters. 4.3. Experimental Setup For the intention selection task, we post-train two variants of Qwen3-VL (Bai et al., 2025), namely the 2B and 4B models, using a consistent supervised fine-tuning (SFT) configuration. We adopt a per-device batch size of 1 with gradient accumulation over 8 steps to maintain stable optimization under memory constraints. The models are trained for 2 epochs with a learning rate of 2Ă10â52Ă 10^-5 and a warmup ratio of 0.1. We use the AdamW optimizer implemented in PyTorch, with bfloat16 (bf16) precision enabled and fp16 disabled. For the intention goal proposal task, we employ a larger Qwen3-VL 8B model, as this task requires generating diverse and distinguishable candidate intentions rather than selecting from predefined options. Despite the increased model capacity, we retain the same training configuration as in the selection setting to ensure comparability and stability. The larger model capacity enables improved generative performance, which is critical for producing high-quality and semantically distinct candidate goals. 4.4. Metrics and Baselines For intention detection and instance detection, which are formulated as classification tasks (i.e., selecting the correct answer from a set of candidates), we adopt accuracy as the primary evaluation metric. Accuracy measures the proportion of correctly predicted labels over the total number of samples and is standard for multiple-choice video question answering benchmarks. As baselines for intention recognition, we compare our approach against a diverse set of models reported in the IntentQA (Li et al., 2023) benchmark, including EVQA, CoMem, HGA, HME, HQGA, and VGT, as well as Blind GPT and VideoQA, which are very variant in terms of size and methodology(e.g. LLM-based and Graph-based). The human performance on the benchmark and the details of the experiment are reported in (Li et al., 2023). However, for the Inst-IT benchmark, we use the state of the art video-language models that are in the same category as our models in terms of size. These models include LLaVA-NeXT-Video, ShareGPT4Video, LLaVA-OV (SI), LLaVA-OV, LLaVA-Video, InternVL2, and LLaVA-NeXT-Inst-IT (Peng et al., 2024). For the open-vocabulary comparison setting, where no goal candidate are provided and the model must generate responses conditioned on video content, we formulate the task as a free-form video question answering problem. We consider three evaluation settings as baselines: (1) direct answer generation, where the model produces an answer without intermediate reasoning; (2) answer selection, where the model selects from a set of candidate answers; and (3) our proposed approach, which first generates candidate goals (intentions) and then selects the most appropriate one. Each setting is evaluated in both zero-shot and fine-tuned regimes. To assess the quality of generated answers, we employ complementary automatic metrics. ROUGE-1 measures unigram overlap, while ROUGE-L captures similarity based on the longest common subsequence between generated and reference texts (Barbella and Tortora, 2022). Cosine Similarity (CosSim) is computed between sentence embeddings to evaluate semantic similarity at the representation level (Chandrasekaran and Mago, 2021). Finally, BERTScore F1 leverages contextualized embeddings from pretrained language models to measure token-level semantic alignment, providing robustness beyond surface-form matching (Hanna and Bojar, 2021). 5. Results As shown in Table 1, our experiments on the IntentVQA benchmark demonstrate that the proposed selection model built on a 4B parameter version of Qwen3-VL achieves substantial improvements over baseline methods. Specifically, the model attains an average accuracy of approximately 80% across both validation and test sets, representing a relative improvement of around 30% compared to existing baselines. Notably, the model also matches human performance, highlighting its effectiveness in intention understanding tasks. Figure 6 presents an ablation study evaluating the impact of model scale (4B vs. 2B parameters) and training regime (fine-tuned vs. zero-shot). The results show that increasing model capacity consistently improves performance. Furthermore, fine-tuning on IntentVQA training examples significantly boosts accuracy compared to zero-shot settings, particularly when the model is required to select from predefined options. This suggests that exposure to task-specific data enhances the modelâs ability to generalize to both validation and test scenarios. These findings demonstrate the strong capability and potential of forward-inverse modeling with video-language models for complex reasoning tasks such as intention detection. However, as shown in Table 2, performance degrades substantially in the open-vocabulary setting where candidates are not provided. In this scenario, the zero-shot model exhibits suboptimal behavior, achieving less than 20 points in terms of ROUGE score, despite maintaining relatively high contextual similarity as measured by BERTScore F1. Furthermore, fine-tuning the model on direct-answer supervision does not alleviate this issue and, in fact, leads to a decrease across all evaluated metrics. Intention options typically introduce constraints, disambiguation, and additional context that effectively reduce the solution space, enabling the model to exploit structural shortcuts. When these options are removed and the model is fine-tuned solely on questionâanswer pairs, inference becomes an unconstrained generation task with a substantially larger output space. This mismatch in task formulation degrades performance, as the model is not adequately trained to operate in such open-ended settings. In contrast, applying our proposed two-stage method yields a consistent improvement, increasing performance by approximately 5 points on average. On the other hand, as shown in Figure 7(a), our experiments on the InsIT benchmark indicate that training on the IntentQA dataset for intention recognition does not lead to catastrophic forgetting. The performance of the fine-tuned model remains very close to its zero-shot counterpart, suggesting that task-specific adaptation preserves generalization ability. Furthermore, as illustrated in Figure 7(b), both models achieve competitive performance compared to models specifically trained on the InsIT benchmark, despite being approximately half their size. This highlights the efficiency of our approach in maintaining strong cross-task performance while using comparatively smaller models. These results establish structured intention recognition as a key component for enabling video-language foundation models to support more socially aware and cognitively capable robotic systems. Safe and Responsible Innovation Statement While our system is designed to improve human-robot collaboration, inferring human intent from video introduces privacy risks, as continuous monitoring may be perceived as surveillance. We acknowledge that models trained on limited public egocentric datasets may reflect cultural biases, potentially underperforming for underrepresented populations. Deployment in assistive or caregiving contexts demands particular care to avoid misinterpretation of vulnerable individualsâ behavior. Conclusion This paper introduced IntentVLM, a novel two-stage video-language framework designed for open-vocabulary human intention recognition. The approach is inspired by forward-inverse modeling in cognitive science by decomposing intention understanding into goal candidate generation followed by structured inference through selection. Experimental results on IntentQA and Inst-IT Bench confirm that IntentVLM significantly outperforms existing baselines and matches human-level performance while maintaining strong generalization across external tasks. Our work demonstrates that structured reasoning enables high-level cognitive inference in multimodal models. References M. Abrini, O. Abend, D. Acklin, H. Admoni, G. Aichinger, N. Alon, Z. Ashktorab, A. Atreja, M. Auron, A. Aufreiter, et al. (2025) Proceedings of 1st workshop on advancing artificial intelligence through theory of mind. arXiv preprint arXiv:2505.03770. Cited by: §2. S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. (2025) Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §2, §4.2, §4.3. C. L. Baker, R. Saxe, and J. B. Tenenbaum (2009a) Action understanding as inverse planning. Cognition 113 (3), p. 329â349. Cited by: §2. C. L. Baker, R. Saxe, and J. B. Tenenbaum (2009b) Action understanding as inverse planning. Cognition 113 (3), p. 329â349. External Links: Document Cited by: §2. M. Barbella and G. Tortora (2022) Rouge metric evaluation for text summarization techniques. Available at SSRN 4120317. Cited by: §4.4. M. E. Bratman (1989) Intention and personal policies. Philosophical Perspectives. External Links: Document Cited by: §2. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei (2020) Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS â20. External Links: ISBN 9781713829546 Cited by: §2. M. BrĂŒne, M. Abdel-Hamid, C. LehmkĂ€mper, and C. Sonntag (2007) Mental state attribution, neurocognitive functioning, and psychopathology: what predicts poor social competence in schizophrenia best?. Schizophrenia research 92 (1-3), p. 151â159. Cited by: §1. S. Buyukgoz, G. J., M. Chetouani, and A. Saffiotti (2022) Two ways to make your robot proactive: reasoning about human intentions or reasoning about possible futures. Frontiers in Robotics and AI. External Links: Document Cited by: §1, §2. D. Chandrasekaran and V. Mago (2021) Evolution of semantic similarityâa survey. Acm Computing Surveys (Csur) 54 (2), p. 1â37. Cited by: §4.4. E. Charniak and R. Goldman (1991) A probabilistic model of plan recognition. In Proceedings of the Ninth National Conference on Artificial Intelligence - Volume 1, AAAIâ91, p. 160â165. External Links: ISBN 0262510596 Cited by: §2. M. Chetouani (2026) Introduction to computational human-ai collaboration. In Handbook of Human-AI Collaboration, M. Chetouani, A. Nowak, and P. Lukowicz (Eds.), External Links: ISBN 978-3-031-61050-9 Cited by: §1, §1. Y. Chiu, B. Chang, T. Chen, and C. Yang (2021) Multi-modal user intent classification under the scenario of smart factory. In Proceedings of the AAAI Conference on Artificial Intelligence, External Links: Document Cited by: §2. A. J. CrĂ©tides, M. Abrini, H. Rahimi, and M. Chetouani (2026) Encoding predictability and legibility for style-conditioned diffusion policy. arXiv preprint arXiv:2603.16368. Cited by: §1. J. P. de Ruiter and C. Cummins (2013) Forward modelling requires intention recognition and non-impoverished predictions. Behavioral and Brain Sciences 36 (4), p. 393â394. Cited by: §2. H. L. Dreyfus (2007) Detachment, involvement, and rationality: are we essentially rational animals?. Human Affairs 17 (2), p. 101â109. External Links: Document Cited by: §2. J. Gao, B. Sarkar, F. Xia, T. Xiao, J. Wu, B. Ichter, A. Majumdar, and D. Sadigh (2024) Physically grounded vision-language models for robotic manipulation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 12462â12469. External Links: Document Cited by: §2. C. Grislain, H. Rahimi, O. Sigaud, and M. Chetouani (2025) I-failsense: towards general robotic failure detection with vision-language models. arXiv preprint arXiv:2509.16072. Cited by: §1. D. Hadfield-Menell, S. J. Russell, P. Abbeel, and A. D. Dragan (2016) Cooperative inverse reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), p. 3909â3917. Cited by: §2. M. Hanna and O. Bojar (2021) A fine-grained analysis of bertscore. In Proceedings of the Sixth Conference on Machine Translation, p. 507â517. Cited by: §4.4. M. K. Ho and T. L. Griffiths (2022a) Cognitive science as a source of forward and inverse models of human decisions for robotics and control. Annual Review of Control, Robotics, and Autonomous Systems 5 (1), p. 33â53. Cited by: §1. M. K. Ho and T. L. Griffiths (2022b) Cognitive science as a source of forward and inverse models of human decision making. Psychological Review. Cited by: §2, §3.1. G. Hoffman, T. Bhattacharjee, and S. Nikolaidis (2024) Inferring human intent and predicting human action in humanârobot collaboration. Annual Review of Control, Robotics, and Autonomous Systems 7 (1), p. 73â95. Cited by: §1, §1, §1. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al. (2022) Lora: low-rank adaptation of large language models.. Iclr 1 (2), p. 3. Cited by: §4.2. W. Huang, P. Abbeel, D. Pathak, and I. Mordatch (2022a) Language models as zero-shot planners: extracting actionable knowledge for embodied agents. International Conference on Machine Learning. Cited by: §2. W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, P. Sermanet, T. Jackson, N. Brown, L. Luu, S. Levine, K. Hausman, and brian ichter (2022b) Inner monologue: embodied reasoning through planning with language models. In 6th Annual Conference on Robot Learning, Cited by: §2. Z. Huang, J. Pohovey, A. Yammanuru, and K. Driggs-Campbell (2024) LIT: large language model driven intention tracking for proactive human-robot collaboration â a robot sous-chef application. arXiv preprint arXiv:2406.13787. Cited by: §2. S. Jain and B. Argall (2019) Probabilistic human intent recognition for shared autonomy in assistive robotics. ACM Transactions on Human-Robot Interaction (THRI) 9 (1), p. 1â23. Cited by: §1. H. A. Kautz and J. F. Allen (1986) Generalized plan recognition. In Proceedings of the Fifth AAAI National Conference on Artificial Intelligence, AAAIâ86, p. 32â37. Cited by: §1, §2. M. T. Khan and A. Waheed (2025) Foundation model driven robotics: a comprehensive review. arXiv preprint arXiv:2507.10087v1. Cited by: §2. N. Lesh and O. Etzioni (1995) A sound and fast goal recognizer. In Proceedings of the 14th International Joint Conference on Artificial Intelligence - Volume 2, IJCAIâ95, p. 1704â1710. External Links: ISBN 1558603638 Cited by: §2. J. Li, P. Wei, W. Han, and L. Fan (2023) Intentqa: context-aware video intent reasoning. In Proceedings of the IEEE/CVF international conference on computer vision, p. 11963â11974. Cited by: Figure 4, Figure 4, §4.1, §4.4, Table 1, Table 1. Z. Li, X. Wu, H. Du, F. Liu, H. Nghiem, and G. Shi (2025) A survey of state of the art large vision language models: alignment, benchmark, evaluations and challenges. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), p. 1578â1597. External Links: Document Cited by: §2. Z. Lian, H. Sun, L. Sun, H. Chen, L. Chen, H. Gu, Z. Wen, S. Chen, S. Zhang, H. Yao, B. Liu, R. Liu, S. Liang, Y. Li, J. Yi, and J. Tao (2025) OV-mer: towards open-vocabulary multimodal emotion recognition. In Proceedings of the 42nd International Conference on Machine Learning, ICMLâ25. Cited by: §2. Y. Lin, S. F. Ghoreishi, T. Lan, and M. Imani (2025) Reinforcement learning for human-AI collaboration via probabilistic intent inference. In Reinforcement Learning Conference, External Links: Link Cited by: §2. N. Madan, A. MĂžgelmose, R. Modi, Y. S. Rawat, and T. B. Moeslund (2024) Foundation models for video understanding: a survey. arXiv preprint arXiv:2405.03770. Cited by: §1. J. MalĂ©cot, H. Rahimi, J. Cattoni, M. Samson, M. Abrini, M. Khoramshahi, M. Pino, and M. Chetouani (2026) HARMONI: multimodal personalization of multi-user human-robot interactions with llms. arXiv preprint arXiv:2601.19839. Cited by: §1. P. Masters and S. Sardina (2019) Cost-based goal recognition in navigational domains. J. Artif. Int. Res. 64 (1), p. 197â242. External Links: ISSN 1076-9757, Document Cited by: §2. B. Nageris, F. Meneguzzi, and R. Mirsky (2024) Goal recognition using actor-critic optimization. arXiv preprint arXiv:2501.01463. Cited by: §2. A. Y. Ng and S. J. Russell (2000) Algorithms for inverse reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning (ICML), p. 663â670. Cited by: §2. T. Nguyen, D. Hsu, W. Lee, T. Leong, L. Kaelbling, T. Lozano-Perez, and A. Grant (2011) Capir: collaborative action planning with intention recognition. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 7, p. 61â66. Cited by: §1. W. Peng, L. Meng, Y. Chen, Y. Xie, Y. Liu, T. Gui, H. Xu, X. Qiu, Z. Wu, and Y. Jiang (2024) INST-it: boosting instance understanding via explicit visual prompt instruction tuning. arXiv preprint arXiv:2412.03565. Cited by: Figure 5, Figure 5, §4.1, §4.4. Z. Qian, M. Kryven, T. Gao, and J. B. Tenenbaum (2021) Modeling human intention inference from visual motion using inverse planning. arXiv preprint arXiv:2112.00903. Cited by: §2. A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever (2021) Learning transferable visual models from natural language supervision. Cited by: §2. H. Rahimi, M. Abrini, M. Khoramshahi, and M. Chetouani (2025a) User-vlm: llm contextualization with multimodal pre-trained user models. In ToM4AI@ 39th Annual AAAI Conference on Artificial Intelligence, Cited by: §2. H. Rahimi, M. Abrini, J. Malecot, Y. Lai, A. Jacquet CrĂ©tides, M. Khoramshahi, and M. Chetouani (2025b) Demographic user modeling for social robotics with multimodal pre-trained models. In Proceedings of the 27th International Conference on Multimodal Interaction, p. 337â343. Cited by: §2. H. Rahimi, A. Bahaj, M. Abrini, M. Khoramshahi, M. Ghogho, and M. Chetouani (2025c) User-vlm 360: personalized vision language models with user-aware tuning for social human-robot interactions. In Proceedings of the 27th International Conference on Multimodal Interaction, p. 326â336. Cited by: §2. H. Rahimi, J. Cattoni, M. Beghili, M. Abrini, M. Khoramshahi, M. Pino, and M. Chetouani (2025d) Reasoning llms for user-aware multimodal conversational agents. In 2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), p. 443â448. Cited by: §2. M. RamĂrez and H. Geffner (2009) Plan recognition as planning. In Proceedings of the 21st International Joint Conference on Artificial Intelligence, IJCAIâ09, p. 1778â1783. Cited by: §2. M. RamĂrez and H. Geffner (2010) Probabilistic plan recognition using off-the-shelf classical planners. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAIâ10, p. 1121â1126. Cited by: §2. M. Sap, R. Le Bras, D. Fried, and Y. Choi (2022) Neural theory-of-mind? on the limits of social intelligence in large LMs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Y. Goldberg, Z. Kozareva, and Y. Zhang (Eds.), p. 3762â3780. External Links: Document Cited by: §2. O. C. Schrempf, D. Albrecht, and U. D. Hanebeck (2007) Tractable probabilistic models for intention recognition based on expert knowledge. In 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 3122â3127. External Links: Document Cited by: §2. G. Sukthankar, C. Geib, H. H. Bui, D. Pynadath, and R. P. Goldman (2014) Plan, activity, and intent recognition: theory and practice. Morgan Kaufmann. Cited by: §1, §2. S. Thellman, M. De Graaf, and T. Ziemke (2022) Mental state attribution to robots: a systematic review of conceptions, methods, and findings. ACM Transactions on Human-Robot Interaction (THRI) 11 (4), p. 1â51. Cited by: §1. S. Trick, D. Koert, J. Peters, and C. A. Rothkopf (2019) Multimodal uncertainty reduction for intention recognition in human-robot interaction. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 7009â7016. Cited by: §1. van Den Broek, M. Koch, and T. B. Moeslund (2024) What is proactive human-robot interaction?-a review of a progressive field and its definitions. ACM Transactions on Human-Robot Interaction 13 (4), p. 1â30. Cited by: §1. F. A. Van-Horenbeke and A. Peer (2021) Activity, plan, and goal recognition: a review. Frontiers in Robotics and AI 8. External Links: Document Cited by: §1, §2. Y. Wan, Y. Wu, Y. Wang, J. Mao, and N. Jaques (2025) Infer humanâs intentions before following natural language instructions. AAAIâ25/IAAIâ25/EAAIâ25. External Links: ISBN 978-1-57735-897-8, Link, Document Cited by: §2. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus (2022) Emergent abilities of large language models. Transactions on Machine Learning Research. Note: Survey Certification External Links: ISSN 2835-8856, Link Cited by: §2. L. Ying, K. Jha, S. Aarya, J. B. Tenenbaum, A. Torralba, and T. Shu (2024) GOMA: proactive embodied cooperative communication via goal-oriented mental alignment. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 7099â7106. External Links: Document Cited by: §2. L. Z, M. Y, S. Z, S. S, S. J, and Z. J (2021) Intention understanding in humanârobot interaction based on visual-nlp semantics. Frontiers in Neurobotics. External Links: Document Cited by: §2. D. Zhang, T. Feng, L. Xue, Y. Wang, Y. Dong, and J. Tang (2025a) Parameter-efficient fine-tuning for foundation models. arXiv preprint arXiv:2501.13787. Cited by: §4.2. H. Zhang, Z. Wang, Q. Lyu, Z. Zhang, S. Chen, T. Shu, B. Dariush, K. Lee, Y. Du, and C. Gan (2025b) COMBO: compositional world models for embodied multi-agent cooperation. In ICLR, External Links: Link Cited by: §2. J. Zhang, J. Huang, S. Jin, and S. Lu (2024) Vision-language models for vision tasks: a survey. IEEE transactions on pattern analysis and machine intelligence 46 (8), p. 5625â5644. Cited by: §1. J. Zhao, Y. Wen, Q. Li, M. Hu, Y. Zhou, J. Xue, J. Wu, Y. Gao, Z. Wen, J. Tao, et al. (2025) Deep learning approaches for multimodal intent recognition: a survey. arXiv preprint arXiv:2507.22934. Cited by: §1. T. Ziemke (2020) Understanding robots. Science Robotics 5 (46), p. eabe2987. Cited by: §1.