Paper deep dive
Cooperative Informative Sensing for Monitoring Dynamic Indoor Environments via Multi-Agent Reinforcement Learning
Kanghoon Lee, Matthew M. Sato, Jinnyeong Yang, Seungro Lee, Sujin Lee, Jiachen Li, Kuk-Jin Yoon, Jinkyoo Park, Kincho H. Law, Yoonjin Yoon
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 6/21/2026, 6:22:32 AM
Summary
The paper proposes a learning-based framework for cooperative informative sensing in dynamic indoor environments using Multi-Agent Reinforcement Learning (MARL). The approach addresses the limitations of traditional coverage-based multi-robot monitoring by optimizing for the accuracy of human-centric monitoring tasks (e.g., occupancy and flow patterns) under partial observability. The framework utilizes a decentralized control structure with a specialized neural architecture featuring set-based observation encoding to handle variable numbers of humans and a dual-stage recurrent interaction memory (Ego-GRU and Interaction-GRU) to facilitate inter-robot coordination. The method is trained using Multi-Agent Proximal Policy Optimization (MAPPO) and demonstrates superior performance in maintaining monitoring accuracy compared to classical coverage and persistent monitoring baselines.
Entities (8)
Relation Signals (4)
Dual-Stage Recurrent Interaction Memory → contains → Ego GRU
confidence 100% · composed of two gated recurrent unit (GRU) components: an ego GRU (GRU_ego) followed by an interaction GRU (GRU_int)
Mobile Robot → uses → 2D LiDAR
confidence 100% · Each robot agent i is equipped with a 2D LiDAR sensor
Multi-Agent Reinforcement Learning (MARL) → isusedtotrain → Set-Based Observation Encoding
confidence 90% · We propose a learning-based framework... using multi-agent reinforcement learning (MARL)... The architecture is illustrated from the perspective of a robot ii. IV-A Set-Based Observation Encoding
Multi-Agent Proximal Policy Optimization (MAPPO) → optimizes → Decentralized Partially Observable Markov Decision Process (Dec-POMDP)
confidence 85% · We formulate the... problem as a Dec-POMDP... We then detail the training procedure based on MAPPO
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Monitoring human activity in indoor environments is important for applications such as facility management, safety assessment, and space utilization analysis. While mobile robot teams offer the potential to actively improve observation quality, existing multi-robot monitoring and active perception approaches typically rely on coverage or visitation based objectives that are weakly aligned with the accuracy requirements of human-centric monitoring tasks. In this work, we formulate cooperative active observation as a decentralized control problem in which multiple robots adjust their motion to directly optimize monitoring accuracy under partial observability. We propose a learning-based framework for cooperative policies from decentralized observations using multi-agent reinforcement learning (MARL), supported by an architecture that handles variable numbers of humans and temporal dependencies. Simulation results across diverse indoor environments and monitoring tasks show that the proposed approach consistently outperforms classical coverage, persistent monitoring, and learning-free multi-robot baselines, while remaining robust to changes in the number of observed humans.
Tags
Links
- Source: https://arxiv.org/abs/2604.23179v1
- Canonical: https://arxiv.org/abs/2604.23179v1
Trouble viewing inline? Open PDF directly →
Full Text
47,394 characters extracted from source content.
Expand or collapse full text
Cooperative Informative Sensing for Monitoring Dynamic Indoor Environments via Multi-Agent Reinforcement Learning Kanghoon Lee⋆, Matthew M. Sato⋆, Jinnyeong Yang, Seungro Lee, Sujin Lee, Jiachen Li, Kuk-Jin Yoon, Jinkyoo Park, Kincho H. Law, Yoonjin Yoon† ⋆Both authors contributed equally to this research.†Corresponding author.K. Lee, J. Yang, S. Lee, K. Yoon, J. Park, and Y. Yoon are with the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea. leehoon, jinnyeong6118, roy.seungro.lee, kjyoon, jinkyoo.park, yoonjin@kaist.ac.kr.M. Sato and K. Law are with Stanford University, Stanford, CA, USA. satomm, law@stanford.edu.S. Lee is with the University of California, Berkeley, Berkeley, CA, USA. sujin.lee@berkeley.edu.J. Li is with the University of California, Riverside (UCR), CA, USA. jiachen.li@ucr.edu. Abstract Monitoring human activity in indoor environments is important for applications such as facility management, safety assessment, and space utilization analysis. While mobile robot teams offer the potential to actively improve observation quality, existing multi-robot monitoring and active perception approaches typically rely on coverage or visitation based objectives that are weakly aligned with the accuracy requirements of human-centric monitoring tasks. In this work, we formulate cooperative active observation as a decentralized control problem in which multiple robots adjust their motion to directly optimize monitoring accuracy under partial observability. We propose a learning-based framework for cooperative policies from decentralized observations using multi-agent reinforcement learning (MARL), supported by an architecture that handles variable numbers of humans and temporal dependencies. Simulation results across diverse indoor environments and monitoring tasks show that the proposed approach consistently outperforms classical coverage, persistent monitoring, and learning-free multi-robot baselines, while remaining robust to changes in the number of observed humans. I Introduction Understanding how humans utilize indoor spaces is important for facility managers, developers, and engineers, enabling better building reconfiguration and informing the design of new buildings to improve efficiency and user experience [9]. Accurate observations of human presence, movement, and interactions are essential for stakeholders, providing the data to estimate occupancy and activity patterns within the space. While many indoor environments already feature sensing infrastructure such as fixed cameras, indoor geometry (walls and corridors) imposes strong constraints on observation, leading to frequent occlusions. Because human motion is dynamic and complex [18], continuous observation from fixed viewpoints is challenging, resulting in sparse and fragmented observations that leave large portions of the environment unobserved for prolonged intervals. As a result, human-centric monitoring tasks that require accurate and continuous observations cannot be adequately addressed by static sensing systems or a single mobile robot. To address sensing limitations, a variety of approaches have been studied in active perception [2], informative path planning [4], multi-robot coverage [5], and persistent monitoring [19]. Classical informative path planning methods typically optimize trajectories to maximize information gain or coverage, often assuming static or slowly varying processes. Similarly, multi-robot coverage and persistent monitoring strategies emphasize uniform or frequent visitation over time, without explicitly accounting for the distribution of human behaviors. While effective for tasks such as environmental mapping or long-term surveillance, these approaches are less suited for monitoring dynamic human environments, where the relevance of observations evolves over time. Moreover, existing methods generally emphasize visitation or observability metrics, rather than directly optimizing for the accuracy of downstream human-centric estimates derived from partial observations, which is the focus of this work. Figure 1: Illustration of cooperative active observation for human-centric monitoring in indoor environments. Red/gray dots denote visible/non-visible human agents, whose observability is limited by indoor occlusions and the robots’ restricted fields of view (FoV). Mobile robots actively control their motion to acquire observations within their FoV (shaded blue regions), supporting human persistent monitoring at the individual, zone, and inter-zone levels under partial observability. In this work, we reframe active observation in indoor environments from traditional coverage or visitation objectives to a multi-agent decision-making framework using multiple mobile robots. Specifically, robots must determine where and when to observe to reduce uncertainty in task-relevant quantities derived from latent human dynamics, such as human-centric monitoring at the individual, zone, and inter-zone levels as shown in Figure˜1. We use cooperative sensing to improve the accuracy of downstream human activity estimates under partial observability. This formulation naturally captures the coupling between sensing actions and estimation quality, providing a unified framework for diverse human-centric monitoring tasks. To address the challenges of partial observability and dynamic human motion, we develop a learning-based framework for cooperative monitoring with multiple mobile robots. Since the problem involves decentralized observations and motion control, with success measured by the accuracy of human-centric monitoring, we adopt a multi-agent reinforcement learning (MARL) approach to learn cooperative policies in a closed-loop manner. Our policy architecture specifically supports variable-sized human observations and temporal dependencies, allowing robust operation regardless of the number of humans present. We evaluate the proposed approach through simulation studies and demonstrate consistent performance improvements over classical coverage, persistent monitoring, and learning-free multi-robot navigating approaches across a diverse set of monitoring tasks. In summary, the main contributions are as follows: • We formalize a cooperative informative sensing problem under partial observability that supports multiple monitoring tasks in dynamic indoor environments. • We propose a learning-based multi-robot monitoring framework with a network architecture that effectively handles partial and variable-sized human observations, while enabling scalable inter-robot coordination. • We validate the proposed approach by demonstrating improved performance over diverse multi-robot sensing methods across multiple monitoring tasks. I Related Works I-A Active Observation and Information Gathering Active observation studies how an agent selects sensing actions such as viewpoints or trajectories to improve the quality of acquired information [2]. Classical robotics approaches include frontier-based exploration that move robots toward the boundary between known and unknown space [25], and information-theoretic planning methods that maximize expected information gain [4]. In multi-robot settings, these principles have been extended to coordinated sensing and team-level information gathering under practical constraints such as limited communication or resources [13, 6]. Recently, deep RL and MARL have been applied to active observation for viewpoint control in target tracking [10], adaptive multi-agent tracking in dynamic environments [17], and cooperative coverage and exploration [5]. In our work, we address the cooperative sensing problem for dynamic indoor monitoring with multiple robots. Our approach optimizes auxiliary monitoring objectives derived from a shared belief state, enables transfer across monitoring tasks, and supports integration with existing fixed sensing infrastructure. (a) (b) (c) Figure 2: An illustration of environment. (a) A generated map showing the room and corridor structure, as well as distinct colored zones. (b) Synthetically generated human trajectories by a random hierarchical planner, avoiding the shaded buffer region to ensure safe paths. (c) An example of the simulation with robots and humans; the shaded blue regions represent the visible field of view (FoV) for each robot, as computed by the visibility function fvisf_vis. I-B Multi-Robot Navigation and Coverage Multi-robot navigation and coverage have been studied under a variety of task objectives that emphasize coordinated motion and spatial coverage. One line of work studies multi-robot coverage path planning (MCPP), which coordinates robot paths to guarantee complete coverage of the environment while optimizing a makespan [21]. Some extensions add objectives such as regional priorities [15] or turn minimization [22]. Persistent monitoring (PM) moves beyond single-pass coverage, designing periodic trajectories to ensure bounded revisit times or sustained observation frequency over space [19, 12]. Other task settings include search-and-rescue, which emphasizes efficient search over large or uncertain regions [24], perimeter defense or surveillance with spatiotemporal guarding responsibilities [11, 23], and encirclement tasks that focus on coordinated positioning around a target [7]. Exploration methods instead focus on frontier expansion in the observed workspace [5]. While these approaches address diverse navigation objectives, they largely optimize coverage or task execution metrics. In contrast, our work focuses on cooperative informative sensing, explicitly optimizing robot motion to improve human-centric monitoring accuracy under partial observability. I Problem Formulation We formulate the multi-robot cooperative informative sensing problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP, [3]). The objective is to learn cooperative sensing policies for a team of robots to accurately predict key measures associated with dynamic human agents in an indoor environment, such as occupancy or flow patterns. Each robot agent is equipped with a 2D LiDAR sensor for obstacle detection, including indoor structures, and a front-facing camera for human detection. The process is defined by the tuple ⟨ℐ,,,,ℛ,,γ⟩ ,S, O, A,R,T,γ . I-1 Agent ℐI denotes the set of n decision-making robot agents, written as ℐ≡ℐR=1,…,nI _R=\1,…,n\. The environment also contains m non-controllable human agents ℐH=1,…,mI_H=\1,…,m\, who follow their own distinct paths. I-2 State A state st∈s_t represents the complete configuration at timestep t, including the indoor environment layout and all agent states. The indoor layout Ω⊂ℝ2 ^2 represents the static and collision-free workspace. The kinematic state for each agent i∈ℐR∪ℐHi _R _H consists of its pose (pti,θti)(p_t^i, _t^i) and linear speed vti∈[0,vmaxi]v_t^i∈[0,v_ ^i]. Here, pti∈Ωp_t^i∈ is the x-yx-y position and θti∈[0,2π) _t^i∈[0,2π) is the orientation. The state for each robot agent additionally includes the measurement received by the 2D LiDAR sensor for that robot, dti∈ℝLd_t^i ^L, which is a vector of distances representing the L LiDAR beams. Therefore, the overall global state is defined as: st=(Ω,(pti,θti,vti,dti)i∈ℐR,(ptj,θtj,vtj)j∈ℐH).s_t=( ,\(p_t^i, _t^i,v_t^i,d_t^i)\_i _R,\(p_t^j, _t^j,v_t^j)\_j _H). (1) I-3 Observation =×i∈ℐRi O=×_i _RO_i is the set of joint observations of all robot agents. Each robot agent i∈ℐRi _R receives a local observation oti∈io_t^i ^i, which includes the precise pose of agent i, the agent’s LiDAR measurements, and noisy measurements of currently visible human agents. Visibility of human j∈ℐHj _H from robot i∈ℐRi _R within the layout Ω is determined by the visibility function: fvis(pti,θti,ptj,Ω)∈0,1.f_vis(p_t^i, _t^i,p_t^j, )∈\0,1\. (2) The visibility function returns 11 if human j is visible to robot i, accounting for sensor constraints and environmental occlusions based on the robot pose (pti,θti)(p_t^i, _t^i) and human position ptjp_t^j. Let ℐH,tiI_H,t^i be the set of indices of humans visible to robot i at time t: ℐH,ti=j∈ℐH∣fvis(pti,θti,ptj,Ω)=1.I_H,t^i=\j _H f_vis(p_t^i, _t^i,p_t^j, )=1\. (3) Then, the local observation otio_t^i is defined as follows: oti=(Ω,(pti,θti,vti,dti)i∈ℐR,(p~tj,θ~tj,v~tj)j∈ℐH,ti),o_t^i=( ,\(p_t^i, _t^i,v_t^i,d_t^i)\_i _R,\( p_t^j, θ_t^j, v_t^j)\_j _H,t^i), (4) where (p~tj,θ~tj,v~tj)( p_t^j, θ_t^j, v_t^j) is the noisy measurement of the true pose (ptj,θtj)(p_t^j, _t^j) and velocity vtjv_t^j of visible human j, assumed drawn from a sensor noise model, e.g., ((ptj,θtj),Σsensor)N((p_t^j, _t^j), _sensor). I-4 Action =×i∈ℐRi A=×_i _RA^i is the set of joint actions of all robot agents. The individual action ati∈ia_t^i _i for robot i controls its desired linear speed vcmd∈0.0,1.0,2.0 m/sv_cmd∈\0.0,1.0,2.0\ m/s and steering command δcmd∈−π8,0.0,+π8 rad/s _cmd∈\- π8,0.0,+ π8\ rad/s. The selected action serves as input to the low-level controller. I-5 Reward Across tasks, we maintain a persistent team belief btb_t over human-related latent variables, which is updated using newly acquired observations at time t. Each task defines a deterministic estimator Φ(⋅) (·) that maps the current belief to a task-level measure mt=Φ(bt)m_t= (b_t). Active improvement is quantified as the change in this measure after the belief update as follows: rt=∥mt−mt−1∥1.r_t= m_t-m_t-1 _1. (5) This reward encourages behaviors that actively refine task-relevant estimates through informative observations, rather than optimizing a static state-based objective. As a concrete example, in the human persistent monitoring task, Φ is the identity mapping so that mtm_t corresponds to the persistent belief state. Then, the reward becomes the total belief update magnitude. During training, the environment internally accesses ground-truth human states to update btb_t, while the policy receives only local observations. I-6 Transition Function :×→P()T:S×A→ P(S) determines the probability distribution over the next state st+1s_t+1 given the current state sts_t and joint action ta_t. The transition model operates with a discrete time interval Δt t. Human agents j∈ℐHj _H follow their predefined distinct paths, determined by an external process independent of robot actions ta_t. For each robot agent i∈ℐRi _R, its state is updated based on its current state (pti,θti,vti)(p_t^i, _t^i,v_t^i) and the selected action ati=(vcmdi,δcmdi)a_t^i=(v^i_cmd,δ^i_cmd). The low-level controller updates the actual linear speed towards the command vcmdiv^i_cmd, resulting in vt+1iv_t+1^i. The position and orientation are updated deterministically based on the current state and steering command δcmdiδ^i_cmd as follows: θt+1i=θti+δcmdi⋅Δt,pt+1i,x=pti,x+vticos(θti)⋅Δt,pt+1i,y=pti,y+vtisin(θti)⋅Δt, \ aligned _t+1^i&= _t^i+δ^i_cmd· t,\\ p_t+1^i,x&=p_t^i,x+v_t^i ( _t^i)· t,\\ p_t+1^i,y&=p_t^i,y+v_t^i ( _t^i)· t, aligned . (6) where pti=(pti,x,pti,y)p_t^i=(p_t^i,x,p_t^i,y). The overall next state st+1s_t+1 combines the static layout Ω and the updated states of all agents. The objective is to find an optimal joint policy π∗=πi∗i∈ℐRπ^*=\ _i^*\_i _R that maximizes the expected discounted cumulative reward, where γ∈[0,1)γ∈[0,1) denotes the discount factor: π∗=argmax[∑t=0∞γtrt].π^*= πargmax\;E [ _t=0^∞γ^tr_t ]. (7) IV Learning Cooperative Sensing Policies This section describes the proposed MARL framework for the problem formulated in Section˜I. We first present the overall network architecture used in our actor-critic formulation, including the set-based observation encoding, dual-stage recurrent interaction memory, decision making, and value estimation modules, as shown in Figure˜3. We then detail the training procedure based on multi-agent proximal policy optimization (MAPPO, [26]), outlining the optimization of the shared policy and centralized critic. Figure 3: Architecture of the proposed network. (Left) Set-based observation encoding produces per-robot features via permutation-invariant attention. (Middle) A dual-stage recurrent interaction memory combines an ego GRU with attention-based inter-robot communication. (Right) The resulting features are used for per-robot decision making and centralized value estimation. The architecture is illustrated from the perspective of a robot i. IV-A Set-Based Observation Encoding Each robot i∈ℐRi _R receives a local observation otio_t^i at every timestep t, as defined in Eq.˜4, consisting of its ego state (pti,θti,vti,dti)(p_t^i, _t^i,v_t^i,d_t^i) and a variable-sized set of visible human states (p~tj,θ~tj,v~tj)j∈ℐH,ti\( p_t^j, θ_t^j, v_t^j)\_j _H,t^i. Due to the varying number and ordering of visible humans across time and robots, human observations are modeled as an unordered set and encoded using a permutation-invariant architecture [14]. The ego state is embedded by an ego multilayer perceptron (MLP) into hrob,ti∈ℝdh_rob,t^i ^d, and each human state is embedded by a shared MLP into token features of the same dimension. The tokens are then stacked into Hhum,ti∈ℝNti×dH_hum,t^i ^N_t^i× d with Nti=|ℐH,ti|+1N_t^i=|I_H,t^i|+1, where the additional learnable dummy token ensures a valid output even when no human is observed. We aggregate human tokens via cross-attention using a multi-head attention (MHAMHA) network with the ego embedding as the query and the human tokens as keys and values, resulting in a context vector chum,ti∈ℝdc_hum,t^i ^d as follows: chum,ti=MHA(hrob,ti,Hhum,ti,Hhum,ti).c_hum,t^i=MHA\! (h_rob,t^i,\;H_hum,t^i,\;H_hum,t^i ). (8) This design preserves permutation invariance and supports variable-sized sets via attention masking, without requiring persistent human identities or explicit tracking across timesteps. The concatenated representation hrob,ti∥chum,tih_rob,t^i\,\|\,c_hum,t^i is then passed to the recurrent memory module. IV-B Dual-Stage Recurrent Interaction Memory To capture temporal dependencies while enabling inter-robot coordination, we maintain a dual-stage recurrent memory composed of two gated recurrent unit (GRU) components: an ego GRU (GRUegoGRU_ego) followed by an interaction GRU (GRUintGRU_int). The ego GRU updates a private state from the concatenated per-robot representation as follows: hego,ti=GRUego(hego,t−1i,hrob,ti∥chum,ti).h_ego,t^i=GRU_ego\! (h_ego,t-1^i,\;h_rob,t^i\,\|\,c_hum,t^i ). (9) Robots then exchange information by attending over the collection of ego hidden states. Let Hego,t=[hego,t1;…;hego,tn]∈ℝn×dH_ego,t=[h_ego,t^1;…;h_ego,t^n] ^n× d denote the stacked ego states. For each robot i, we compute a coordination context vector via multi-head attention with hego,tih_ego,t^i as the query and the stacked ego states as keys and values: cego,ti=MHA(hego,ti,Hego,t,Hego,t).c_ego,t^i=MHA\! (h_ego,t^i,\;H_ego,t,\;H_ego,t ). (10) Finally, the interaction GRU integrates this coordination signal over time to form an interaction-aware memory: hint,ti=GRUint(hint,t−1i,hego,ti∥cego,ti).h_int,t^i=GRU_int\! (h_int,t-1^i,\;h_ego,t^i\,\|\,c_ego,t^i ). (11) This factorization preserves a robot-specific temporal state while incorporating coordination information through attention, enabling scalable multi-robot interaction without requiring explicit pairwise tracking or fixed ordering. IV-C Decision Making and Value Estimation The interaction-aware representation hint,tih_int,t^i is used for per-robot decision making, while value estimation is performed in a centralized manner. The policy maps hint,tih_int,t^i to independent categorical distributions over the linear and angular velocity action sets defined in Section˜I. For a centralized value estimation, the interaction states are stacked as Hint,t=[hint,t1;…;hint,tn]∈ℝn×dH_int,t=[h_int,t^1;…;h_int,t^n] ^n× d, and a learnable query vector qV∈ℝdq_V ^d attends to this set via multi-head attention to obtain a pooled global representation: zt=MHA(qV,Hint,t,Hint,t),z_t=MHA\! (q_V,\;H_int,t,\;H_int,t ), (12) which is subsequently mapped through an MLP to produce a scalar value estimate VtV_t. IV-D Multi-Agent Reinforcement Learning We train the proposed architecture using MAPPO [26] under centralized training with a shared policy πθ _θ across all robots and a separately parameterized centralized critic VϕV_φ. While each robot executes actions based on its own interaction-aware representation, value estimation is performed centrally during training. Policy updates follow the clipped PPO objective with generalized advantage estimation (GAE) for variance reduction. Let A^ti A_t^i denote the advantage estimate for robot i and rti(θ)=πθ(ati∣oti)πθold(ati∣oti)r_t^i(θ)= _θ(a_t^i o_t^i) _ _old(a_t^i o_t^i) the probability ratio. The objective function is defined as follows: ℒ(θ,ϕ)=t[ (θ,φ)=E_t [ min(rti(θ)A^ti,clip(rti(θ),1−ϵ,1+ϵ)A^ti) (r_t^i(θ) A_t^i,\,clip(r_t^i(θ),1-ε,1+ε) A_t^i ) (13) −cvℒV(ϕ)+ceℋ(πθ)]. -c_vL_V(φ)+c_eH( _θ) ]. where ℒV(ϕ)L_V(φ) is the value regression loss and ℋ(πθ)H( _θ) denotes an entropy regularizer. To stabilize centralized value estimation, the critic is trained with access to ground-truth human states. Visibility indicators are augmented as one-hot encodings, and all humans are assumed observable when computing value targets. This augmentation is used only during training and does not affect execution, where each robot relies solely on its local observation. V Experiments V-A Experimental Setup Figure 4: Cooperative informative sensing performance comparison. Average errors in human tracking (left), zone occupancy (middle), and human flow (right) monitoring tasks across different numbers of robots (3–5). Lower is better. Error bars indicate standard deviation. TABLE I: Experimental Setup and Implementation Details Environment & Sensing Map / #Rooms / #Zones 80×4080× 40 m / 12 / 7 Time step / horizon Δt=1 t=1 s, T=500T=500 #Humans / #Robots M=20M=20, N=5N=5 Observation noise σp=0.2 _p=0.2 m, σθ=0.1 _θ=0.1 rad Action space v∈0,1,2v∈\0,1,2\ m/s (max 2.0) ω∈−π/8,0,π/8ω∈\-π/8,0,π/8\ rad/s Tracking sensor range 10 m, FoV 90∘90 , occlusion via ray sampling (K=5K=5) LiDAR range 10 m, FoV 360∘360 , 16 rays Network Architecture & MAPPO Training Network hidden dim 64 Learning rate 3×10−43× 10^-4 (Actor / Critic) MAPPO / Optimization clip 0.2, max grad norm 0.5, value coef 0.5 Discount / GAE γ=0.99γ=0.99, λ=0.95λ=0.95 Entropy regularization 0.01 (linear speed), 0.001 (rotation) Rollout / Updates 250 parallel environments, 1000 steps per update, 20 minibatches, 20 epochs, 50 chunk length 50M timesteps To evaluate cooperative informative sensing policies, we develop a simplified 2D simulation environment. We generate indoor environments with room and corridor structures [16], avoiding unrealistic cave-like topologies. We sequentially carve randomly sized, overlapping rooms and connect their centroids with corridors to ensure the entire map Ω is connected. Physically overlapping rooms are grouped into distinct zones as shown in Figure˜2a. These environments are then populated with synthetic human paths generated using a hierarchical model. A high-level planner selects goal destinations via a Markov chain [20] across the rooms and samples realistic dwell times from a log-normal distribution [1]. A low-level executor then generates plausible, collision-free motion by following an A∗ path [8] with a pure pursuit controller, as shown in Figure˜2b. Robot partial observations are implemented via ray sampling, which approximates occlusions by checking K parallel points to avoid slow sequential ray-tracing. An example of the whole environment is shown in Figure˜2c. The complete experimental configuration, including environment parameters and MARL training hyperparameters, is summarized in Table˜I. V-B Description of Monitoring Tasks and Baselines We evaluate our methods on three indoor monitoring tasks that capture complementary aspects of indoor human activity. Each task emphasizes a different sensing granularity: tracking individual humans, estimating zone-level occupancy, and capturing flow between zones. As defined in Section˜I, the team maintains a belief and derives a task-level estimate mtm_t under partial observability. The magnitude of belief updates defines the training reward, and the task-specific estimators are specified below. Let Z denote the number of semantic zones presented in Figure˜2a. • Human Persistent Monitoring: We maintain a persistent per-human position belief p^tj∈ℝ2 p^j_t ^2 for each human j. If any robot observes human j at time t, p^tj p^j_t is updated to the current position and otherwise remains unchanged. The task-level estimate is mt=[p^t1;…;p^tM]m_t=[ p^1_t;…; p^M_t]. • Zone Occupancy Monitoring: We maintain zone-level occupancy estimates derived from the persistent position belief p^tj p^j_t. Each estimate is mapped to a zone, and the task-level estimate mt∈ℝZm_t ^Z is defined as the zone-wise occupancy count vector, mt[z]=∑j=1M[p^tj∈zone z]m_t[z]= _j=1^MI[ p^j_t z]. • Human Flow Monitoring: We maintain a per-human zone-transition belief (z^tj,prev,z^tj,cur,τ^tj)( z^j,prev_t, z^j,cur_t, τ^j_t), updated only when an observed human changes zones. Upon a transition, we record the previous and current zones and the transition time. The task-level estimate mt∈ℝZ×Zm_t ^Z× Z is the directed flow count over all zone pairs, defined as mt[z1,z2]=∑j=1M[z^tj,prev=z1∧z^tj,cur=z2]m_t[z_1,z_2]= _j=1^MI[ z^j,prev_t=z_1 z^j,cur_t=z_2]. For all tasks, the per-step reward component is clipped to a fixed range to prevent extreme updates and stabilize training. We compare our MARL-based active observation policy with baseline planners, covering both classical and learning-free approaches to evaluate the impact of learned coordination on information gathering efficiency. • FC (Fixed Cameras) uses static sensors placed to cover the largest visible zones without overlap, serving as a reference for non-mobile sensing performance. Sensors are placed sequentially using a greedy algorithm to maximize the area not yet covered by previously placed static sensors. • WS (Waypoint Sampling) imitates human high-level planning by randomly selecting goal points within and across rooms and navigating between them sequentially, resulting in spatially diverse and natural exploration behavior. • MCPP (Multi-Robot Coverage Path Planning) uses a spanning tree coverage strategy [21] to achieve efficient coverage of the environment. The tree is partitioned among robots, with path endpoints connected to form closed loops that are repeatedly traversed while minimizing makespan. • PM (Persistent Monitoring) uses the solution of a linear program to control robot velocities along closed paths, guaranteeing a minimum bounded frequency that a space is observed by the robots [19]. V-C Cooperative Informative Sensing Performance We evaluate the proposed cooperative informative sensing algorithm by comparing it with baseline methods across the three monitoring tasks described in Section˜V-B. To assess performance under different sensing resources, we vary the number of robots from 3 to 5 while keeping the environment and human dynamics fixed. The proposed MARL policy is trained with five robots and evaluated in a zero-shot manner for different team sizes (e.g., n=3,4n=3,4), and the same trained model is used for all subsequent experiments. Figure˜4 shows the comparative performance across methods and team sizes, with results averaged over 250 scenarios. Overall, the proposed MARL policy achieves the lowest average error across all tasks and evaluated team sizes, including zero-shot settings. As the number of robots increases, the errors generally decrease for all methods and the performance gaps across algorithms become smaller, indicating that additional sensing resources reduce overall difficulty. A notable observation is that FC exhibits substantially larger standard deviations in human tracking and zone occupancy, suggesting that its mean performance is more sensitive to scenario variations. PM exhibits a non-monotonic trend in human flow estimation, where the error slightly increases from 2.162.16 at n=3n=3 to 2.182.18 at n=4n=4, indicating that additional robots do not always lead to improved coordination. V-D Robustness under Out-of-Distribution (OOD) Scenario Figure 5: Evaluation under OOD scenarios. Average tracking error across the default setting and four OOD variants involving changes in population size and human movement distributions (OOD cases shown in red). We evaluate if the proposed MARL policy maintains strong monitoring performance under distribution shifts that were not observed during training. To this end, we construct four OOD scenarios that perturb either the number of humans or their behavioral dynamics while keeping the sensing and environment configurations unchanged. Specifically, we vary the human population from the default 20 to 10 (Sparse) and 30 (Crowded), increase dwell times to induce more persistent stationary behavior (Long Dwell), and amplify the transition probability toward a spatially isolated right-side zone (zone 3 in Figure˜2a) by 22 times (Skewed). As shown in Figure˜5, most methods show only minor variations under Sparse, Crowded, and Long Dwell, indicating that moderate shifts in population size or dwell-time persistence do not significantly affect monitoring performance. In contrast, performance decreases under the Skewed scenario for all methods. It suggests that visitation concentrated in spatially isolated regions is more difficult to monitor. FC shows the largest drop, suggesting limited adaptability when such regions become more frequently used. In comparison, our MARL policy achieves the lowest tracking error across all scenarios, including Skewed, demonstrating consistent performance under distribution shifts. V-E Scalability and Marginal Utility Analysis Figure 6: Scalability and marginal utility analysis. Tracking error (left) and marginal performance gain (right) with increasing number of robots. To evaluate scalability and the benefit of additional robots, we vary the number of robots from 1 to 10 while keeping the environment and human dynamics fixed. Figure˜6 shows the average tracking error (left) and the marginal utility (right) as the number of robots increases. We define the marginal utility at team size n as the reduction in tracking error when increasing the number of robots from n−1n-1 to n as follows: Δ(n)=E(n−1)−E(n), (n)=E(n-1)-E(n), (14) where E(n)E(n) denotes the average tracking error with n robots. Overall, performance improves as more robots are deployed, indicating that additional sensing resources enhance spatial coverage, while the marginal utility quickly saturates as the team size grows. In this map configuration, the marginal gain becomes small beyond approximately 55 robots, showing diminishing returns as coverage redundancy increases. FC achieves the lowest tracking error at larger team sizes (e.g., n≥8n≥ 8), reflecting the advantage of globally optimized fixed placements that minimize overlap. In contrast, mobile sensing approaches exhibit less consistent scaling behavior, suggesting that coordination overhead limits the effective utilization of larger teams. Notably, PM shows a slight performance degradation at n=8n=8, indicating instability at higher team sizes. The MARL policy maintains consistently strong performance across all team sizes without significant degradation, demonstrating robust scalability. V-F Cooperation with Existing Sensing Systems We study cooperation with an existing fixed sensing system by integrating fixed cameras with MARL-controlled mobile robots under a fixed sensing budget. Concretely, we consider several hybrid integration scenarios with a total budget of five sensors, ranging from fixed-only (F5+M0F5+M0) to mobile-only (F0+M5F0+M5), including mixed configurations such as F2+M3F2+M3 and F1+M4F1+M4. We trained a MARL policy for each configuration. Figure˜7 summarizes the results, where the leftmost bar corresponds to FC (F5+M0F5+M0) and the rightmost bar corresponds to MARL (F0+M5F0+M5). Figure 7: Cooperation with an existing sensing system. Average tracking error under hybrid integration of fixed cameras (F) and mobile robots controlled by MARL policy (M) with a total budget of five sensors. Horizontal lines indicate standalone baselines. (a) (b) Figure 8: Visibility heatmap under hybrid integration. Observed-region heatmaps for the mobile-only setting (F0+M5, top) and the hybrid setting (F1+M4, bottom). Red denotes regions observed by MARL-controlled mobile robots, and blue denotes the fixed camera FoV. The results show that hybrid integration can outperform both fixed-only and mobile-only sensing. In particular, mixed configurations such as F2+M3F2+M3 and F1+M4F1+M4 achieve lower tracking error than using only fixed cameras or only mobile robots. Importantly, F1+M4F1+M4 performs comparably to F0+M5F0+M5 while incurring lower operational cost, since fixed infrastructure is typically cheaper to operate than an additional mobile robot. This behavior is further illustrated in Figure˜8, where the observed-region heatmaps differ between F0+M5F0+M5 and F1+M4F1+M4. In the hybrid setting, mobile robots avoid regions already covered by the fixed camera and allocate their trajectories to complementary areas, showing adaptive coverage distribution. However, as the number of fixed cameras further increases, performance begins to degrade. When fixed sensors dominate the sensing budget, mobile robots have limited opportunity to contribute additional coverage, reducing the benefit of adaptive coordination. These results suggest that an appropriate balance between fixed and mobile sensing is necessary to maximize information gain, since learning an effective cooperative strategy is more difficult with a large number of robots. V-G Impact of Fixed Camera Placement in the Hybrid Setting To understand where to install a single FC in the hybrid setting (F1+M4), we evaluate seven feasible camera placements in the same indoor layout as shown in Figure˜9 and compare their tracking performance in Table˜I. Overall, placements located in outer regions (1, 3, and 7) of the map achieve the best mean performance. These placements cover areas that would otherwise require explicit robot visits. It reduces unnecessary traversal and allows the robots to concentrate on monitoring the remaining parts of the environment. However, Placement 6 does not provide comparable improvement despite being away from the center, as it covers only a single traffic room, as shown in Figure˜2a. In contrast, Placement 7 remains highly effective by covering a room at the end of a long corridor that is costly for robots to reach. Placement 7 also reduces the standard deviation, indicating improved robustness by consistently observing a frequently traversed yet hard-to-reach area. On the other hand, central placements (2, 4, and 5) perform worse, as robots inevitably traverse the central region, making the FC largely redundant. These results highlight that effective FC deployment should consider the indoor layout topology and robot traversal characteristics. In particular, placement plays a critical role in improving monitoring performance under a constrained sensing budget. Figure 9: Fixed camera placement candidates (F1+M4). We consider seven feasible camera placement IDs (1-7) in the same indoor layout. Each shaded region indicates the fixed camera FoV under that placement. TABLE I: Tracking Error Across Camera Position ID Position Index 1 2 3 4 5 6 7 Mean 6.72 7.79 6.83 7.33 8.46 7.57 6.76 Std 4.38 5.13 4.25 4.89 5.83 5.02 3.80 V-H Reward Analysis To evaluate whether the proposed reward aligns with actual monitoring performance, we analyze the correlation between episode reward and tracking error under a fixed scenario while varying robot behaviors. For WS, behavioral diversity is introduced by sampling different human-following trajectories, and for MARL, multiple runs are conducted with different random seeds and stochastic policies. In contrast, FC, MCPP, and PM are deterministic and therefore correspond to single points in Figure˜10. We observe a clear negative correlation between reward and tracking error (R=−0.653R=-0.653), with even stronger correlations for occupancy (R=−0.80R=-0.80) and flow (R=−0.89R=-0.89). Although the reward is not a perfect surrogate for the evaluation metric, the strong negative correlations indicate that increasing reward is consistently associated with improved monitoring performance. Figure 10: Correlation between reward and tracking error. Each point represents a different robot behavior evaluated in the same scenario. The dashed line denotes linear regression and the shaded region indicates the 95% confidence interval. VI Conclusions and Discussions In this paper, we present a MARL-based cooperative informative sensing framework for dynamic indoor monitoring. Our approach optimizes observation accuracy across diverse human-centric tasks and highlights the importance of cost-effective hybrid integration with existing fixed infrastructure. In particular, FC placement must consider indoor layout topology, as redundant coverage can reduce monitoring gains. While we evaluate in a 2D simulation to enable large-scale statistical validation, our architecture processes low-dimensional semantic states which guarantees that the cooperative policy remains structurally invariant when integrating real-world perception pipelines such as RGB-D object tracking. Future work will focus on deploying this modular framework in high-fidelity 3D simulators and real-world environments. We also aim to incorporate richer semantic information and complex human-centric constraints. In addition, we plan to leverage the reasoning capabilities of Large Language Models (LLMs) or Vision-Language Models (VLMs) to enhance agent explainability and generalizability. Acknowledgement This research was supported by Center for Advanced Urban Systems (CAUS) of Korea Advanced Institute of Science and Technology (KAIST) funded by GS E&C. This work was also supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.RS-2025-00517342). This material is based upon work supported by the National Science Foundation (NSF) Graduate Research Fellowship Program under Grant No. DGE-2146755. This research is also partially supported by the Stanford’s Center for Sustainable Development and Global Competitiveness (SDGC). References [1] D. Ashbrook and T. Starner (2003) Using gps to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing 7 (5), p. 275–286. Cited by: §V-A. [2] R. Bajcsy (1988) Active perception. Proceedings of the IEEE 76 (8), p. 966–1005. Cited by: §I, §I-A. [3] D. S. Bernstein, R. Givan, N. Immerman, and S. Zilberstein (2002) The complexity of decentralized control of markov decision processes. Mathematics of Operations Research 27 (4), p. 819–840. Cited by: §I. [4] F. Bourgault, A. A. Makarenko, S. B. Williams, B. Grocholsky, and H. F. Durrant-Whyte (2002) Information based adaptive robotic exploration. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 1, p. 540–545. Cited by: §I, §I-A. [5] J. Chiun, S. Zhang, Y. Wang, Y. Cao, and G. Sartoretti (2025) MARVEL: multi-agent reinforcement learning for constrained field-of-view multi-robot exploration in large-scale environments. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 11392–11398. Cited by: §I, §I-A, §I-B. [6] S. Choudhury, N. Gruver, and M. J. Kochenderfer (2020) Adaptive informative path planning with multimodal sensing. In Proceedings of the International Conference on Automated Planning and Scheduling, Vol. 30, p. 57–65. Cited by: §I-A. [7] F. Gou, H. Du, C. Zhao, and Y. Cai (2025) A policy-guided reinforcement learning method for encirclement control in multiobstacle environment. IEEE Transactions on Neural Networks and Learning Systems. Cited by: §I-B. [8] P. E. Hart, N. J. Nilsson, and B. Raphael (1968) A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics 4 (2), p. 100–107. Cited by: §V-A. [9] M. A. Hassanain (2010) Analysis of factors influencing office workplace planning and design in corporate facilities. Journal of Building Appraisal 6 (4), p. 183–197. Cited by: §I. [10] H. Jeong, H. Hassani, M. Morari, D. D. Lee, and G. J. Pappas (2021) Deep reinforcement learning for active target tracking. In 2021 IEEE International Conference on Robotics and Automation (ICRA), p. 1825–1831. Cited by: §I-A. [11] H. Kim, K. Lee, J. Park, J. Li, and J. Park (2025) Human implicit preference-based policy fine-tuning for multi-agent reinforcement learning in usv swarm. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vol. , p. 18653–18659. Cited by: §I-B. [12] X. Lan and M. Schwager (2013) Planning periodic persistent monitoring trajectories for sensing robots in gaussian random fields. In 2013 IEEE International Conference on Robotics and Automation (ICRA), p. 2415–2420. Cited by: §I-B. [13] M. Lauri, E. Heinänen, and S. Frintrop (2017) Multi-robot active information gathering with periodic communication. In 2017 IEEE International Conference on Robotics and Automation (ICRA), p. 851–856. Cited by: §I-A. [14] J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh (2019) Set transformer: a framework for attention-based permutation-invariant neural networks. In Proceedings of the 36th International Conference on Machine Learning, p. 3744–3753. Cited by: §IV-A. [15] K. Lee, H. Kim, J. Li, and J. Park (20262026) Priority-aware multi-robot coverage path planning. IEEE Robotics and Automation Letters 11 (3), p. 3534–3541. Cited by: §I-B. [16] P. Merrell, E. Schkufza, and V. Koltun (2010) Computer-generated residential building layouts. In ACM SIGGRAPH Asia 2010 papers, p. 1–12. Cited by: §V-A. [17] J. Rückin, L. Jin, and M. Popović (2022) Adaptive informative path planning using deep reinforcement learning for uav-based active sensing. In 2022 International Conference on Robotics and Automation (ICRA), p. 4473–4479. Cited by: §I-A. [18] A. Rudenko, L. Palmieri, M. Herman, K. M. Kitani, D. M. Gavrila, and K. O. Arras (2020) Human motion trajectory prediction: a survey. The International Journal of Robotics Research 39 (8), p. 895–935. Cited by: §I. [19] S. L. Smith, M. Schwager, and D. Rus (2011) Persistent robotic tasks: monitoring and sweeping in changing environments. IEEE Transactions on Robotics 28 (2), p. 410–426. Cited by: §I, §I-B, 4th item. [20] C. Song, Z. Qu, N. Blumm, and A. Barabási (2010) Limits of predictability in human mobility. Science 327 (5968), p. 1018–1021. Cited by: §V-A. [21] J. Tang, C. Sun, and X. Zhang (2021) MSTC*: multi-robot coverage path planning under physical constrain. In 2021 IEEE International Conference on Robotics and Automation (ICRA), p. 2518–2524. Cited by: §I-B, 3rd item. [22] I. Vandermeulen, R. Groß, and A. Kolling (2019) Turn-minimizing multirobot coverage. In 2019 International Conference on Robotics and Automation (ICRA), p. 1014–1020. Cited by: §I-B. [23] S. Velhal, S. Sundaram, and N. Sundararajan (2022) A decentralized multirobot spatiotemporal multitask assignment approach for perimeter defense. IEEE Transactions on Robotics 38 (5), p. 3085–3096. Cited by: §I-B. [24] J. Wu, L. Cheng, S. Chu, and Y. Song (2024) An autonomous coverage path planning algorithm for maritime search and rescue of persons-in-water based on deep reinforcement learning. Ocean Engineering 291, p. 116403. Cited by: §I-B. [25] B. Yamauchi (1997) A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97.’Towards New Computational Principles for Robotics and Automation’, p. 146–151. Cited by: §I-A. [26] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu (2022) The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems 35, p. 24611–24624. Cited by: §IV-D, §IV.