Paper deep dive
HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answering
Ji Li, Bo Wang, Jing Xia, Mingyi Li, Shiyan Hu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/21/2026, 3:29:27 AM
Summary
The paper introduces HIMM, a non-parametric memory framework for embodied agents that disentangles episodic and semantic memory. It uses visual reasoning to recall episodic experiences and a program-style rule extraction mechanism to create reusable semantic memory, achieving state-of-the-art performance on embodied question answering benchmarks.
Entities (8)
Relation Signals (6)
HIMM → uses → Episodic Memory
confidence 95% · Our approach recalls episodic experiences based on semantic similarity and verifies them through visual reasoning
HIMM → uses → Semantic Memory
confidence 95% · we introduce a program-style rule extraction mechanism that converts experiences into structured, reusable semantic memory
HIMM → evaluatedon → A-EQA
confidence 92% · yielding a 7.3% gain in LLM-Match and an 11.4% gain in LLM MatchXSPL on A-EQA
HIMM → evaluatedon → GOAT-BENCH
confidence 92% · as well as +7.7% success rate and +6.8% SPL on GOAT-Bench.
Semantic Memory → createdby → Program-Style Rule Extraction
confidence 90% · we introduce a program-style rule extraction mechanism that converts experiences into structured, reusable semantic memory
Episodic Memory → verifiedby → Visual Reasoning
confidence 90% · verifies them through visual reasoning on exploration maps
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deploying Multimodal Large Language Models as the brain of embodied agents remains challenging, particularly under long-horizon observations and limited context budgets. Existing memory assisted methods often rely on textual summaries, which discard rich visual and spatial details and remain brittle in non-stationary environments. In this work, we propose a non-parametric memory framework that explicitly disentangles episodic and semantic memory for embodied exploration and question answering. Our retrieval-first, reasoning-assisted paradigm recalls episodic experiences via semantic similarity and verifies them through visual reasoning, enabling robust reuse of past observations without rigid geometric alignment. In parallel, we introduce a program-style rule extraction mechanism that converts experiences into structured, reusable semantic memory, facilitating cross-environment generalization. Extensive experiments demonstrate state-of-the-art performance on embodied question answering and exploration benchmarks, yielding a 7.3% gain in LLM-Match and an 11.4% gain in LLM MatchXSPL on A-EQA, as well as +7.7% success rate and +6.8% SPL on GOAT-Bench. Analyses reveal that our episodic memory primarily improves exploration efficiency, while semantic memory strengthens complex reasoning of embodied agents.
Tags
Links
- Source: https://arxiv.org/abs/2602.15513v2
- Canonical: https://arxiv.org/abs/2602.15513v2
Trouble viewing inline? Open PDF directly →
Full Text
41,914 characters extracted from source content.
Expand or collapse full text
HIMM: Human-Inspired Long-Term Memory Modeling for Embodied Exploration and Question Answering Ji Li 1 , Bo Wang 1 , Jing Xia 1 , Mingyi Li 2 , Shiyan Hu 1,† Abstract—Deploying Multimodal Large Language Models as the brain of embodied agents poses chal- lenging when reasoning long-horizon observations. Existing memory-augmented approaches typically compress observations into textual summaries losing fine-grained information, and overlook the inherent differences across memory types. To address these limitations, we propose HIMM, a memory framework that explicitly models episodic and semantic memory for embodied exploration and question answering. Our approach recalls episodic experiences based on se- mantic similarity and verifies them through visual rea- soning on exploration maps, enabling efficient reuse of past observations without strict geometric alignment. Meanwhile, we introduce a program-style rule ex- traction mechanism that transforms experiences into structured, reusable semantic memory, facilitating cross-environment generalization. Our methods out- performs prior methods on on embodied exploration and question answering benchmarks, yielding a 7.3% LLM-Match gain and an 11.4% LLM-Match×SPL gain on A-EQA, as well as +7.7% success rate and +6.8%SPLonGOAT-Bench.Analysesrevealthatour episodic memory primarily improves exploration effi- ciency, while semantic memory strengthens complex reasoning of embodied agents. I. Introduction In recent years, Multimodal Large Language Models (MLLMs) have advanced rapidly, demonstrating strong reasoning and perception capabilities across a wide range of domains. Despite this progress, deploying MLLMs as the core module of embodied agents in real-world environments remains highly challenging. A key difficulty lies in enabling agents to efficiently process long-horizon observations while retaining task-relevant information under limited context lengths. To address this issue, recent works introduce various forms of working mem- ory, allowing agents to retrieve a small subset of past observations instead of conditioning on the entire sensory history [1]–[3]. By coupling task-relevant information retrieval with frontier-based exploration strategies, these methods achieve efficient and effective performance in embodied exploration and question answering. However, most existing approaches remain confined to information within the current episode, limiting their ability to leverage accumulated experience over time. In contrast, real-world embodied agents are expected to 1 The University of Hong Kong, Hong Kong SAR. 2 Beijing Institute of Technology, Beijing, China. † corresponding author. jerichojili, bowang, jingxia@connect.hku.hk, mingyili@bit.edu.cn, shiyanhu@hku.hk continuously integrate observations across episodes and exploit long-term knowledge acquired from past interac- tions. Approaches such as [4], [5] attempt to address this by summarizing cross-episode information into hierarchi- cal textual descriptions that guide future exploration. Meanwhile, [6] introduces cross-episode memory into the reasoning loop. Nevertheless, its task formulation remains restrictive, typically evaluating an MLLM-based information-collection system within a fixed explored space derived from a previous timeline. Despite these efforts, existing memory architectures remain funda- mentally rigid, and they suffer from two major limita- tions. First, this design inevitably discards a substantial amount of spatial and visual cues in raw observations, limiting the agents memory capacity. Second, most methods adopt a single memory representation to model heterogeneous memory systems, which fails to capture the distinct characteristics and functions of different memory types, as extensively studied in cognitive psy- chology [7]. These limitations become even more pronounced when moving from simulation to realistic dynamic environ- ments, where two additional fundamental challenges arise. First, fusion maps across time, whether repre- sented as persistent nodes [2] or voxel grids [8], becomes inherently difficult in the presence of environmental changes and noise. Second, information retrieved from past does not necessarily reflect the current state of the environment. Based on these observations, we argue that embodied agent should neither rely on rigid long- term geometric fusion nor blindly trust past episodic information. Instead, it should recall prior experiences in a soft and associative manner while continuously grounding them in active exploration and perception, closely mirroring how humans reason about and interact with dynamic environments. Motivated by these insights, we propose HIMM, a non-parametric memory framework inspired by human memory systems, which explicitly model episodic and semantic memory for embodied exploration and question answering. We propose a retrieval-first and reasoning- assisted paradigm. Retrieved memory snapshot are se- lectively grounded and verified through visual reasoning, ensuring reliable reuse in long-horizon timeline. Comple- mentarily, in contrast to leveraging environment-specific and free-form textual summaries, our method distills structured, high-level, and reusable semantic memory through pseudocode-like decision workflows and decision arXiv:2602.15513v2 [cs.RO] 2 Mar 2026 Fig. 1: Overview of our proposed HIMM. Episodic memory and semantic memory guide agent’s efficient exploration and accurate reasoning, acrossing sequential cognitive states which governed by meta memory. deviations detection. This dual-system design enables stable long-term knowledge accumulation while support- ing robust exploration and long-horizon reasoning. We summarize our contributions as follows: •We introduce HIMM, a memory framework that explicitly models episodic and semantic memory for embodied agents, enabling self-evolving without additional training. •We propose to ground retrieved episodic memories through visual reasoning over memory snapshots and exploration maps, together with a program-style rule extraction mechanism that enhances complex embodied reasoning. •Our method achieves new state-of-the-art results, with 7.3% and 11.4% performance gain on A- EQA [9], and 7.7%, 6.8% performance gain on GOAT-Bench [10], demonstrating significantly im- proved exploration efficiency and remarkable ques- tion answering capability. I. Related Works A. MLLM-based Embodied Agents Recent proprietary foundation models, such as GPT-5 and Gemini 3, have significantly advanced multi-image understanding by leveraging extended context lengths and large-scale pretraining [11], and enable more compre- hensive perception over long visual sequences, forming a strong backbone for embodied agents [12]–[17]. Beyond vision-language foundation models, imagination driven designs augment observations via generative priors [18], [19], while confidence calibration approaches explicitly model when MLLM should stop by quantifying epistemic uncertainty [20]–[22]. Despite these advances, foundation models still re- main computationally expensive, and naively process- ing uniformly sampled frames is often suboptimal for questions that require selectively identifying informative observations for exploration and reasoning. To address these challenges, [1], [2], [23], [24] introduces CLIP [25] and SAM-based [26] open-vocabulary scene graphs to structure semantics for MLLM-driven exploration. While [3] further incorporates raw images to enrich memory, its key-frame selection remains constrained by detec- tors and lacks explicit geometric modeling, leading to inefficient long-horizon exploration. In contrast to ex- isting geometric-semantic graph methods often depend on heavy multi-frame fusion, our approach marks a shift toward decoupling semantic and geometric spaces, en- abling retrieval-time interaction that supports adaptive cross-episode recalling. B. Experience and Memory for Embodied Agents Research on memory mechanisms for embodied agents seeks to enable the storage, retention, and retrieval of past experiences, facilitating the transition from purely reactive systems to agents capable of maintaining con- text and autonomous adaptation. [27] and [4] adopt descriptive text, temporal and spatial cues to manage continuously growing histories, allowing robots to re- call previously visited regions and reduce redundant exploration. However, memory built primarily on image and video captioning discard a substantial amount of fine-grained visual information embedded in raw obser- vations. [6] further explores long-term memory under dynamic environments by preserving raw image obser- vations. Nevertheless, its task formulation remains re- strictive, typically evaluating an information searching system within a fixed explored space, hinders its appli- cability to realistic exploration scenarios. More recently, [5] focuses on extracting reusable experience from train- ing trajectories. While effective at capturing high-level patterns, the distilled experiences remain tightly coupled to previously environment, limiting their generalization ability and reasoning enhancement. Motivated by these observations, we draw inspiration from human memory systems and propose to disentangle episodic and seman- tic memory, recalling episode to improve exploration efficiency, while following high-level decision rules in decision making. I. Methods Our overall pipeline is illustrated in Fig.1. Given a posed RGB-D observation, our framework constructs semantic and physical space to represent the scene (Sec- tionIII-A), Next, episodic memory is recalled by visual reasoning to guide exploration, while program-sytle ex- tracted semantic memory is fed into LLM to improve rational (SectionIII-B). A. Semantic and Physical Space for Embodied Agents Navigation in embodied environments requires rea- soning jointly over a semantic space and a physical space. Inspired by cognitive science findings suggesting structured interactions between these two spaces [28], we propose to decouple semantic memory from physical geometry, while enabling their interaction through test- time retrieval. Semantic and Physical Space.At each observation step, a set of region-level semantic embeddings and a place-level image embedding are extracted using SAM and CLIP to form hierarchical embedding-based scene graph, following HOV-SG [1]. Global embedding, re- gional embeddings, region 3D-boxes and raw image are stored in a semantic space indexed by camera pose, as depicted in Fig. 2. To interface with downstream tasks, we leverage a LLM to decompose an instruction into three goal components: atarget object, a set ofrelative objects, and a set ofrelative areas. Each component is embedded and used to query region embeddings in the semantic space. Retrieved embeddings are ranked in a priority order, withtarget objectevidence taking precedence, followed byrelative objectsandrelative areas. While the physical space is represented by a lightweight 2D occupied exploration map constructed from depth observations and camera poses. Retrieval-Time Inter-Space Interaction.In con- strast to prior approaches such as [1], which tightly cou- Global Embedding Region Embedding Region 3D-Box Camera Pose Raw Image Viewj Semantic Space of Episode i xKxK Fig. 2: Illustration of our semantic space for embodied agents. ple semantic representations with multi-view geometric fusion, our semantic memory is updated independently of persistent geometry. Specifically, we only link semantic space to the physical space by rendering the camera poses of retrieved observations onto the occupancy grid. The interaction allowing semantic concepts to be flex- ibly grounded without continuous online fusion, and preserves task-relevant semantic cues while keeping the physical representation compact and efficient. B. Human-Inspired Memory for Embodied Exploration Embodied agents are expected to learn and adapt from their past experiences. Episodic and semantic memory play essential roles in human cognition [7]. This inspires that instead of using a single memory representation to model heterogeneous memory systems, it is important to design different construction and retrieval strategies for each type of memory to better capture their distinct functions. Episodic Memory: Visual Reasoning for Episodic Recall.Episodic memory records episode- level experiences in a detailed and temporally grounded manner [7]. Each episodic memory instance consists of both a semantic space and a physical space, which encode the observed scene from complementary perspectives, as described in Section I-A. As a fine-grained memory type, episodic memory preserves information at the granularity of navigation steps. When encountering a new episode, we first retrieve the Top-Kmost similar past observations based on visual similarity to the current observation. To verify episodic correspondence, we further assess whether the retrieved observations originate from nearby locations using an MLLM. After that, embeddings of the current target are used to query the verified episode, and the episode that accumulates the highest number of Top-Kmatches is selected as prior experience. Mapping-based approaches [29] explicitly align and merge memories by transformations estimation, which are highly sensitive to alignment failures and are prone to cascading errors during long-horizon changing and noises. While we humans recall past experiences through associative and semantic similarity rather than rigid metric fusion of memories. Hence, in contrast to these methods, we propose a gentle episodic recall strategy supported by visual reasoning capability. Method Object Rec. Object Loc. Attribute Rec. Spatial Object State Functional World Know. Overall Socratic LLM-based Exploration w/ Frame Captions GPT-4*25.3 – 28.4 – 27.3–37.7 – 47.2– 54.2 – 29.5–35.5 GPT-4o22.0 – 25.0 – 27.3–40.8 – 50.9– 61.8 – 38.4–35.9 Socratic LLM-based Exploration w/ Scene-Graph Captions CG Scene-Graph*25.3 – 16.5 – 29.2–37.0 – 52.2– 46.8 – 37.8–34.4 SVM Scene-Graph*29.0 – 17.2 – 31.5–31.5 – 54.2– 39.8 – 38.9–34.2 LLaVA-1.5*25.0 – 24.0 – 34.1–34.4 – 56.9– 53.5 – 40.6–38.1 Multi-Frame*34.0 – 34.3 – 51.5–39.5 – 51.9– 45.6 – 36.6–41.8 Open-Sourced MLLM-based Exploration 3D-Mem [3] (Qwen2.5-VL) 25.0 13.9 23.8 7.752.629.3 33.3 3.9 57.526.143.8 13.5 37.5 8.4 39.1 14.6 ReEXplore [5] (Qwen2.5-VL)50.631.329.417.849.131.043.115.166.717.247.927.443.120.946.223.0 Ours (Qwen2.5-VL)39.029.528.417.351.332.437.614.458.324.448.524.847.222.343.123.7 Commercial MLLM-based Exploration Explore-EQA* (GPT-4o)44.0 19.6 37.1 29.6 55.3 36.0 42.1 6.6 46.3 9.2 63.2 35.745.5 22.0 46.9 23.4 CG + Frontier* (GPT-4o)45.042.0 32.1 25.0 50.8 35.2 32.9 18.7 68.5 38.4 58.8 42.245.533.5 47.2 33.3 3D-Mem [3] (GPT-4o)35.0 18.8 50.0 37.3 64.356.350.0 24.7 80.0 49.3 50.0 22.1 30.0 21.4 54.4 33.3 ReEXplore [5] (GPT-4o)37.5 21.9 65.645.867.9 46.160.025.3100.053.565.042.4 35.0 23.658.337.3 Ours (GPT-4o)62.052.763.748.668.552.552.236.891.254.166.445.554.544.365.648.7 Human Agent*89.7 – 72.8 – 85.4–84.8 – 97.8– 78.9 – 88.5–85.1 TABLE I: Performance comparison on A-EQA across different question categories. For each category, we report the LLM-Match score (M) and the LLM-MatchSPL score. “CG” denotes ConceptGraphs. Results marked with * are taken from 3D-Mem [3]. Specifically, given the retrieved Top-Kepisodic obser- vations, MLLM first determines whether further explo- ration is required for the current task. I explore = 1,if exploration is required, 0,otherwise. (1) LetOCC r + = (T r ,F r )composes the exploration map, whereT r represent the retrieved camera poses, and F r denotes the set of frontier points. For each frontier f2 F, letρ(f)indicate whether it can lead to unex- plored regions, andd(f,T r )denote its spatial distance to semantically relevant landmarks. The current and pre- vious exploration maps are jointly provided to a MLLM, outputing pruned frontiers in a visual reasoning manner: F ⋆ = f2F jρ(f) = 1g,ifI explore = 1, f2F jd(f,T r )> d min g,ifI explore = 0. (2) Here, another retrieval-time interaction between seman- tic and physical space is expressed through frontiers pruning operation. Through this reasoning-centric ap- proach, episodic memory informs exploration decisions without requiring explicit geometric fusion, resulting in a stable exploration strategy. Semantic Memory: Program-Style Enhanced Rule Extraction.Semantic memory captures long- term, consolidated knowledge distilled from an agent’s past experiences, including reusable knowledge and pre- ferred response patterns [30]. Unlike episodic memories, which rely on fine-grained details and are inherently tied to specific environments, semantic memory aims to extract reusable rules from dilemmas encountered during reasoning execution, with the core objective of captur- ing solutions that are transferable across environments. However, previous experiments [31] reveal that, directly summarizing agents’ reasoning logsS n are often overly entangled with details that fail to reuse and compress essential knowledge. Visual Reasoning on Exploration map when Iexplore= True Task : You are an embodied agent exploring an indoor environment. Based on your previous memory, you NEVER reached a place which has enough information to answer the question. So you need to explore other unvisited places based on previous experience. Definition: Frontier: A point to an unexplored region based on current cognitive map, that could potentially lead to new information for answering the question. Selecting a frontier means that you will further explore that direction. Prune the frontiers that can’t lead to unexplored area: 1. Check whether the CURRENT map and the PREVIOUS exploration map belong to the same house. You should determine this by comparing the layout shapes (black boundaries). If the layouts do NOT correspond to the same house, STOP and do NOT perform any further operation. 2. If they belong to the same house: Compare the CURRENT map with the PREVIOUS exploration map . For each CURRENT frontier: 1. Align its position onto the corresponding location in the PREVIOUS exploration map. 2. whether it is completely enclosed by black boundaries in any map. If YES, prune this CURRENT frontier because it cannot lead to any unknown area . 3. Check whether the reachable explored regions (gray and green areas) connected to that frontier If NO, prune this CURRENT frontier because it cannot lead to any unknown area . Here is previous exploration map: PREVIOUS_MAP Here is current exploration map: CURRENT_MAP Important: - You may exclude one or multiple frontiers. (a) Task : You are an embodied agent exploring an indoor environment. Based on your previous memory, you NEVER reached a place which has enough information to answer the question. So you need to explore other unvisited places based on previous experience. Definition: Frontier: A point to an unexplored region based on current cognitive map, that could potentially lead to new information for answering the question. Selecting a frontier means that you will further explore that direction. Prune the frontiers that can’t lead to unexplored area: 1. Check whether the CURRENT map and the PREVIOUS exploration map belong to the same house. You should determine this by comparing the layout shapes (black boundaries). If the layouts do NOT correspond to the same house, STOP and do NOT perform any further operation. 2. If they belong to the same house: Compare the CURRENT map with the PREVIOUS exploration map . For each CURRENT frontier: 1. Align its position onto the corresponding location in the PREVIOUS exploration map. 2. whether it is completely enclosed by black boundaries in any map. If YES, prune this CURRENT frontier because it cannot lead to any unknown area . 3. Check whether the reachable explored regions (gray and green areas) connected to that frontier If NO, prune this CURRENT frontier because it cannot lead to any unknown area . Here is previous exploration map: PREVIOUS_MAP Here is current exploration map: CURRENT_MAP Important: - You may exclude one or multiple frontiers. Visual Reasoning on Exploration map when Iexplore= False (b) Fig. 3: Prompt templates for visual reasoning on episodic memory recall. To address these limitations, we draw inspiration from [32] and introduce a program-style enhanced rule extrac- tion strategy. Specifically, we prompt an LLM to analyze the execution pseudocode of a task to prioritize high- level structural information. The model first identifies 3D-Mem Ours 3D-Mem Ours 3D-Mem Ours GT Answer: On the kitchen counter top. GT Answer: On the balcony outside. GT Answer: Closed. Task: Where is the umbrella? Task: Where can I get recipes for what to cook? Task: Is the toilet seat open or closed? Fig. 4: Qualitative case study on A-EQA. Compared with 3D-Mem, our method demonstrates accurate cross-episode recall and more efficient exploration. variablesandfunctionsthat govern the workflows, and then organizes them into a pseudocodePthat explicitly captures control flow and decision logic. In constract to [31], [32], where workflows are typically short, em- bodied agent often involves long-horizon execution with extended decision sequences. In this situation, LLMs tend to over-annotate functions with learned rules that are weakly correlated with task success, obscuring the truly critical decision points. To prevent this, we intro- duce decision deviations detection mechanism on robots’ trajectories [33]. Specifically, given the robot stateM t at timesteptand the corresponding ground-truth trajectory x, we compute: h t =d(M t , x),(3) whered(,)measures the Euclidean distance between the agent’s current position and the ground-truth tra- jectory from training set. In constract to previous work using predefined threshold, we interactively set threshold Sto identify decision deviations at timesteptif h t > S^h t−1 S.(4) Then, we randomly stop the detection at thresholdS stop when the numberKof decision deviation is in the range of 3-5 to form setT=ft k g K k=1 . These deviation timesteps signal potential errors or departures from the expected behavior, allowing embodied agent to ‘debug’ the ‘program’. During the rule extraction, we provide the MLLM with the ground truth answerGT,S n ,P,t k together with their corresponding visual observationsI t k : R=MLLM ( GT,S n ,P,f(t k , I t k )g K k=1 ) , R f i :fφ i,1 7!v i,1 , . . . , φ i,n i 7!v i,n i g. (5) Finally, experience semantic knowledge can be ab- stracted into structured, high-level rules using canonical forms such asif–then,situation–suggestionandproblem– solutionkey-value pairs, where each pair belongs to one specificvariablesorfunctions. At the test-time, the question-similarity retrieved se- mantic memory is loaded into the system prompt to constrain decision-making process. Specifically, MiniLM- L6-v2 variant from Sentence-Transformer [34] is used to extract embeddings for querying question in semantic memory, and compute cosine similarity to retrieve the top-K. Meta Memory:We leverage the success of cognitive process modeling [6], [35] and introduce meta memory via cognitive states automatic switching to build the founda- tion of embodied agent. Specifically, pre-defined discrete cognitive states include: exploration, target verification, target approaching, check ready to answer. IV. Experiments A. Active Embodied Question Answering To evaluate the effectiveness of our memory design, we conduct experiments on the A-EQA benchmark [9] which built on habitat [36], [37]. Here, we compare our approach against other MLLM-based baselines on Task: Is the bedroom furnished and ready to live in? w/o Program-Style Enhanced Rule for Semantic Memoryw Program-Style Enhanced Rule for Semantic Memory ..... The bedroom is furnished with a bed that has bedding, a bedside table with a lamp, and a wardrobe with mirrors.These elements indicate that the room is ready for someone to live in. .....Learn from rules that visual evidence includes multiple essential furnishings (e.g., walls, beds, curtain).This room has bedding, a bedside table with a lamp, but its walls are not painted, so the bedroom is not ready to live in. (a) w/o Program-Style Enhanced Rule for Semantic Memoryw Program-Style Enhanced Rule for Semantic Memory ..... The kitchen area is visible, which likely contains cabinets suitable for storing a knife safely.You can keep the knife in one of cabinets in the kitchen to ensure it is stored properly. ..... The kitchen is a common place to store knives for easy access when cooking. The images show a kitchen area with countertops, which are typical places for storing a new knife. This aligns the rule to prioritize locations strongly associated with the target object in common practice. You can keep the new knife in the kitchen, either in a drawer or on a knife holder. Task: where can I keep a new knife I got?" (b) Task: Where is the toy car? w/o Program-Style Enhanced Rule for Semantic Memoryw Program-Style Enhanced Rule for Semantic Memory <No exploration> The toy car is likely in the child's bedroom or playroom. ..... This aligns with the rule to search common areas where toys are typically kept, such as cabinets in living rooms. The direct visual evidence of the toy car allows us to confidently answer the question. This follows the rule from past experience of finalizing the answer when direct, unambiguous evidence of the object and its location is found. The toy car is on the cabinet next to the TV. (c) Fig. 5: Qualitative examples of how program-style enhanced rules for semantic memory impact the reasoning trace for answering questions. We highlight failed reasoning trajectory in red and successful ones in green. the A-EQA benchmark, assessing both exploration ef- ficiency and performance on answering complex open- ended questions. In addition, our semantic memory is distilled from 184 episodes in the training split, with a category distribution aligned with that of the test set. Metrics.We reportLLM-MatchandLLM- Match SPL as evaluation metrics to measure answer quality and exploration efficiency. Specifically, SPL is computed as the ratio between the shortest path length and the actual trajectory length, normalized to[0,1], where a higher value indicates a more efficient navi- gation trajectory.LLM-Matchevaluates the semantic correctness of the agent s answers using GPT-4o as an automated judge.LLM-MatchSPLfurther weights the semantic correctness score by navigation efficiency, jointly reflecting both answer quality and exploration effectiveness. If the agent fails to produce a response, answer is outputted by GPT-4o without visual input, and the SPL term is set to zero. Analysis.Table Ishows that our method surpasses all baselines with the GPT-4o backbone. Specifically, the overall LLM-Match improves from 58.3% to 65.6%, and LLM-Match×SPL increases substantially from 37.3% to 48.7%, indicating both higher answer accuracy and im- proved exploration efficiency. Notably, significant gains are observed across most task categories: Object recognition improves from 45.0% to 62.0%, functional reasoning from 65.0% to 66.4%, and world knowledge from 35.0% to 54.5%. On the one hand, our method achieves higher LLM-Match×SPL across all categories. As further illustrated in Fig. 4, episodic memory recall guided by visual reasoning ef- fectively steers exploration toward goal-relevant regions, MethodSuccess Rate (%)↑SPL (%)↑ Traditional Methods Modular GOAT [10]*24.917.2 Modular CLIP on Wheels [10]*16.110.4 SenseAct-N Skill Chain [10]*29.511.3 SenseAct-N Monolithic [10]*12.36.8 MLLM-based Exploration 3D-Mem [3] † (Qwen2.5-VL-7B-Instruct)49.420.7 ReEXplore [5] † (Qwen2.5-VL-7B-Instruct)53.232.6 Ours † (Qwen2.5-VL-7B-Instruct)53.931.7 Explore-EQA [21] † (GPT-4o)55.037.9 CG w/ Frontier Snapshots [3] † (GPT-4o)61.545.3 3D-Mem w/o Memory [3] † (GPT-4o)58.638.5 3D-Mem [3] † (GPT-4o) 68.948.9 ReEXplore [5] † (GPT-4o)59.842.5 Ours † (GPT-4o)72.856.1 TABLE I: Results on GOAT-Bench under theVal Un- seensplit. “CG” denotes ConceptGraphs [23]. Methods marked with * are reported from GOAT-Bench, while those marked withyuse MLLM-based exploration. resulting in more purposeful navigation. On the other hand, the pronounced improvements in world knowledge and functional recall highlight our semantic memory sup- porting the reasoning-intensive and cognitively complex tasks. Fig.5illustrates how semantic memory constrains the agents reasoning during question answering. Task- relevant semantic memory encapsulates high-level prior experience, reducing the arbitrariness of MLLM reason- ing and constraining the agent to adopt more appropriate strategies, thereby improving answering performance. When switching to the Qwen-2.5-VL-7B-Instruct backbone, our method maintains consistent performance gains. It achieves the second-best LLM-Match and the best LLM-Match×SPL, indicating a favorable balance between answer accuracy and exploration efficiency across different MLLM backbones. In contrast to ReEx- MethodEpisodic Memory Semantic MemoryLLM-Match LLM-Match×SPL Ours (Qwen2.5-VL-7B-Instruct)✓43.123.7 ✓41.623.2 ✓39.117.5 Ours (GPT-4o)✓65.648.7 ✓62.347.2 ✓60.941.1 TABLE I: Ablation studies on our method by selectively disabling episodic or semantic memory. Results are reported with two backbone MLLMs. plore [5], which employs a stronger MLLM backbone but exhibits noticeable performance degradation in object recognition and world knowledge, our framework contin- ues to scale positively with increasing MLLM capability. This suggests that our design more effectively leverages enhanced MLLM reasoning capacity, rather than being bottlenecked by other components in system. B. Life-Long Visual Language Navigation GOAT-Bench [10] is a lifelong visual navigation bench- mark designed around a multi-target navigation task. In this setting, an agent is required to navigate se- quentially to multiple target objects within previously unseen environments. Targets are specified in diverse modalities, including object category names (e.g.,refrig- erator), natural language descriptions (e.g.,toy next to the dresser), or goal images. Following prior baseline methods [3], [5], we evaluate our approach on a1/10-scale subset of the val-unseen split, which contains one episode per scene across 36 scenes, resulting in a total of 278 navigation subtasks. In addition, our semantic memory is constructed on the 36 training scenes. Metrics.Success Rate and Success weighted by Path Length (SPL) is adpoted as evaluation metrics. An episode is deemed successful when the agent termi- nates within 1 m of the navigation target. Similar to OpenEQA, SPL measures efficiency by weighting suc- cessful episodes according to the length of the navigation path. Analysis.As shown in Table I, our method outper- forms most baseline methods, demonstrating its ability to effectively leverage knowledge accumulated from both episode memory and semantic memory, as evidenced by consistently higher Success Rate and SPL scores. Compared with 3D-Mem under the GPT-4o backbone, our method achieves a notable improvement in life-long navigation. The Success Rate increases from 68.9% to 72.8%, and SPL rises from 48.9% to 56.1%, indicating more accurate and efficient goal-directed exploration. Under the Qwen backbone, our method slightly under- performs ReEXplore in SPL metric. We attribute this primarily to the relatively weaker visual reasoning ca- pability of Qwen backbone when operating on semantic- labeled exploration maps, which limits the correctness of frontier pruning. C. Ablation Study TableIIIpresents ablation results on the A-EQA benchmark. When episodic memory is removed, we ob- serve a clear degradation in exploration efficiency. Specif- ically, LLM-Match drops from 41.3% to 39.1% under the Qwen backbone, and from 65.6% to 60.9% under the GPT-4o backbone. This performance decrease indicates that our episodic memory provides effective guidance for exploration by enabling the agent to reuse rele- vant prior experiences. In contrast, removing semantic memory results in a more decline in answer accuracy than corresponding reduction in SPL. This highlights the critical role of semantic memory in preserving trans- ferable knowledge and supporting embodied reasoning during question answering. Overall, these ablation re- sults demonstrate that the two memory modules of- fer complementary benefits: episodic memory primarily improves exploration efficiency, while semantic memory contributes more substantially to accuracy of reasoning- intensive question answering. V. Conclusion We presented HIMM, a novel memory framework for MLLM-based embodied agents that explicitly model episodic and semantic memory to address long-horizon exploration and reasoning. By constructing a visual rea- soning paradigm, our approach enables agents to recall and verify past episodic experiences without relying on rigid geometric fusion, thereby supporting robust cross- episode reuse of observations and improving exploration efficiency. In addition, resuable and structed rules as semantic memory distilled from experiences tackle com- plex reasoning in embodied tasks. Comprehensive eval- uation on A-EQA and GOAT-Bench demonstrates the effectiveness of our memory modeling in improving both exploration efficiency and answering accuracy. References [1]A. Werby, C. Huang, M. Büchner, A. Valada, and W. Burgard, “Hierarchical open-vocabulary 3d scene graphs for language- grounded robot navigation,” inFirst Workshop on Vision- Language Models for Navigation and Manipulation at ICRA 2024, 2024. [2]Z. Yan, S. Li, Z. Wang, L. Wu, H. Wang, J. Zhu, L. Chen, and J. Liu, “Dynamic open-vocabulary 3d scene graphs for long- term language-guided mobile manipulation,”IEEE Robotics and Automation Letters, 2025. [3]Y. Yang, H. Yang, J. Zhou, P. Chen, H. Zhang, Y. Du, and C. Gan, “3d-mem: 3d scene memory for embodied exploration and reasoning,” inProceedings of the Computer Vision and Pattern Recognition, 2025, p. 17294–17303. [4]M. Zhai, Z. Gao, Y. Wu, and Y. Jia, “Memory-centric embod- ied question answer,”arXiv preprint arXiv:2505.13948, 2025. [5]G. Zhang, M. Ding, J. Wu, R. Liao, and V. Tresp, “Reex- plore: Improving mllms for embodied exploration with con- textualized retrospective experience replay,”arXiv preprint arXiv:2511.19033, 2025. [6]M. F. Ginting, D.-K. Kim, X. Meng, A. M. Reinke, B. J. Kr- ishna, N. Kayhani, O. Peltzer, D. Fan, A. Shaban, S.-K. Kim, et al., “Enter the mind palace: Reasoning and planning for long-term active embodied question answering,” inConference on Robot Learning. PMLR, 2025, p. 5072–5106. [7]E. Tulving, “How many memory systems are there?”American psychologist, vol. 40, no. 4, p. 385, 1985. [8]Z. Wang, S. Lee, and G. H. Lee, “Dynam3d: Dynamic layered 3d tokens empower vlm for vision-and-language navigation,” inThe Thirty-ninth Annual Conference on Neural Informa- tion Processing Systems. [9]A. Majumdar, A. Ajay, X. Zhang, P. Putta, S. Yenamandra, M. Henaff, S. Silwal, P. Mcvay, O. Maksymets, S. Arnaud, et al., “Openeqa: Embodied question answering in the era of foundation models,” inProceedings of the IEEE/CVF confer- ence on Computer Vision and Pattern Recognition, 2024, p. 16488–16498. [10]M. Khanna, R. Ramrakhya, G. Chhablani, S. Yenamandra, T. Gervet, M. Chang, Z. Kira, D. S. Chaplot, D. Batra, and R. Mottaghi, “Goat-bench: A benchmark for multi-modal lifelong navigation,” inProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2024, p. 16373–16383. [11]A. K. Lampinen, M. Engelcke, Y. Li, A. Chaudhry, and J. L. McClelland, “Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences,” arXiv preprint arXiv:2509.16189, 2025. [12]A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, and D. Batra, “Embodied question answering,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, p. 1–10. [13]Z. Zhou, Y. Hu, L. Zhang, Z. Li, and S. Chen, “Beliefmapnav: 3d voxel-based belief map for zero-shot object navigation,” arXiv preprint arXiv:2506.06487, 2025. [14]Z. Yang, S. Zheng, T. Xie, T. Xu, B. Yu, F. Wang, J. Tang, S. Liu, and M. Li, “Efficientnav: Towards on-device object- goal navigation with navigation map caching and retrieval,” in The Thirty-ninth Annual Conference on Neural Information Processing Systems. [15]Z. Zhu, X. Wang, Y. Li, Z. Zhang, X. Ma, Y. Chen, B. Jia, W. Liang, Q. Yu, Z. Deng,et al., “Move to understand a 3d scene: Bridging visual grounding and exploration for efficient and versatile embodied navigation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025, p. 8120–8132. [16]S. Saxena, B. Buchanan, C. Paxton, P. Liu, B. Chen, N. Vaske- vicius, L. Palmieri, J. Francis, and O. Kroemer, “Grapheqa: Using 3d semantic scene graphs for real-time embodied ques- tion answering,” inConference on Robot Learning. PMLR, 2025, p. 2714–2742. [17]S. Zhang, D. Huang, J. Deng, S. Tang, W. Ouyang, T. He, and Y. Zhang, “Agent3d-zero: An agent for zero-shot 3d understanding,” inEuropean Conference on Computer Vision. Springer, 2024, p. 186–202. [18]Y. Yang, J. Liu, Z. Zhang, S. Zhou, R. Tan, J. Yang, Y. Du, and C. Gan, “Mindjourney: Test-time scaling with world models for spatial reasoning,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems. [19]Y. Xu, Y. Pan, and Z. Liu, “Dream to recall: Imagination- guided experience retrieval for memory-persistent vision-and- language navigation,”arXiv preprint arXiv:2510.08553, 2025. [20]N. Frahm, P. Patel, Y. Zhang, S. Yu, M. Bansal, and R. Sengupta, “Prune-then-plan: Step-level calibration for sta- ble frontier exploration in embodied question answering,” arXiv preprint arXiv:2511.19768, 2025. [21]A. Z. Ren, J. Clark, A. Dixit, M. Itkina, A. Majumdar, and D. Sadigh, “Explore until confident: Efficient exploration for embodied question answering,” inFirst Workshop on Vision- Language Models for Navigation and Manipulation at ICRA 2024. [22]K. Jiang, Y. Liu, W. Chen, J. Luo, Z. Chen, L. Pan, G. Li, and L. Lin, “Beyond the destination: A novel benchmark for exploration-aware embodied question answering,” inProceed- ings of the IEEE/CVF International Conference on Computer Vision, 2025, p. 9091–9101. [23]Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, et al., “Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning,” in2024 IEEE International Con- ference on Robotics and Automation. IEEE, 2024, p. 5021– 5028. [24]X. Zhou, T. Xiao, L. Liu, Y. Wang, M. Chen, X. Meng, X. Wang, W. Feng, W. Sui, and Z. Su, “Fsr-vln: Fast and slow reasoning for vision-language navigation with hierarchical multi-modal scene graph,”arXiv preprint arXiv:2509.13733, 2025. [25]A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inProceedings of the International Conference on Machine Learning. PmLR, 2021, p. 8748–8763. [26]A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, et al., “Segment anything,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, p. 4015–4026. [27]A. Anwar, J. Welsh, J. Biswas, S. Pouya, and Y. Chang, “Remembr: Building and reasoning over long-horizon spatio- temporal memory for robot navigation,” in2025 IEEE Inter- national Conference on Robotics and Automation. IEEE, 2025, p. 2838–2845. [28]T. Van de Maele, B. Dhoedt, T. Verbelen, and G. Pezzulo, “A hierarchical active inference model of spatial alternation tasks and the hippocampal-prefrontal circuit,”Nature Communica- tions, vol. 15, no. 1, p. 9892, 2024. [29]J. Lazarow, K. Kang, and A. Dehghan, “Rooms from mo- tion: Un-posed indoor 3d object detection as localization and mapping,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems. [30]E. Tulvinget al., “Episodic and semantic memory,”Organi- zation of memory, vol. 1, no. 381-403, p. 1, 1972. [31]J. Liu, Y. Sun, W. Cheng, H. Lei, Y. Chen, L. Wen, X. Yang, D. Fu, P. Cai, N. Deng,et al., “Memverse: Multimodal memory for lifelong learning agents,”arXiv preprint arXiv:2512.03627, 2025. [32]M. Ho, C. Si, Z. Feng, F. Yu, Y. Yang, Z. Liu, Z. Hu, and L. Qin, “Arcmemo: Abstract reasoning composition with lifelong llm memory,”arXiv preprint arXiv:2509.04439, 2025. [33]Z. Yu, Y. Long, Z. Yang, C. Zeng, H. Fan, J. Zhang, and H. Dong, “Correctnav: Self-correction flywheel empow- ers vision-language-action navigation model,”arXiv preprint arXiv:2508.10416, 2025. [34]N. Reimers and I. Gurevych, “Sentence-bert: Sentence em- beddings using siamese bert-networks,” inProceedings of the 2019 conference on empirical methods in natural language pro- cessing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, p. 3982–3992. [35]Y. Cao, J. Zhang, Z. Yu, S. Liu, Z. Qin, Q. Zou, B. Du, and K. Xu, “Cognav: Cognitive process modeling for object goal navigation with llms,” inProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, 2025, p. 9550– 9560. [36]X. Puig, E. Undersander, A. Szot, M. D. Cote, T.-Y. Yang, R. Partsey, R. Desai, A. W. Clegg, M. Hlavac, S. Y. Min,et al., “Habitat 3.0: A co-habitat for humans, avatars and robots,” arXiv preprint arXiv:2310.13724, 2023. [37]S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. Turner, E. Undersander, W. Galuba, A. Westbury, A. X. Chang,et al., “Habitat-matterport 3d dataset (hm3d): 1000 large- scale 3d environments for embodied ai,”arXiv preprint arXiv:2109.08238, 2021.