Paper deep dive
Human-Inspired Pavlovian and Instrumental Learning for Autonomous Agent Navigation
Jingfeng Shan, Francesco Guidi, Mehrdad Saeidi, Enrico Testi, Elia Favarelli, Andrea Giorgetti, Davide Dardari, Alberto Zanella, Giorgio Li Pira, Francesca Starita, Anna Guerra
Intelligence
Status: succeeded | Model: anthropic/claude-sonnet-4.6 | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/24/2026, 1:38:22 AM
Summary
This paper presents a human-inspired hybrid reinforcement learning architecture for autonomous agent navigation that integrates Pavlovian conditioning, Instrumental Model-Free (MF) RL, and Instrumental Model-Based (MB) RL components. Inspired by neuroscience, the framework uses contextual radio cues as conditioned stimuli to shape intrinsic value signals and bias decision-making. A Bayesian arbitration mechanism blends MF and MB estimates based on predicted reliability, while a motivational gate modulates learning based on internal state variables. The approach is validated in a multi-agent target localization task, demonstrating accelerated learning, improved safety, and reduced navigation in high-uncertainty regions compared to standard RL baselines.
Entities (37)
Relation Signals (30)
Jingfeng Shan → affiliatedwith → University of Bologna
confidence 99% · J. Shan, M. Saeidi, E. Testi, E. Favarelli, D. Dardari, and A. Guerra are with WiLab, University of Bologna
Francesco Guidi → affiliatedwith → National Research Council of Italy
confidence 99% · F. Guidi and A. Zanella are with the National Research Council of Italy (CNR)
Alberto Zanella → affiliatedwith → National Research Council of Italy
confidence 99% · F. Guidi and A. Zanella are with the National Research Council of Italy (CNR)
Davide Dardari → affiliatedwith → University of Bologna
confidence 99% · Davide Dardari, Fellow, IEEE University of Bologna, Cesena, Italy
Anna Guerra → affiliatedwith → University of Bologna
confidence 99% · Anna Guerra, Member, IEEE University of Bologna, Cesena, Italy
Giorgio Li Pira → affiliatedwith → University of Bologna
confidence 99% · G. Li Pira and F. Starita are with the University of Bologna, Department of Psychology
Francesca Starita → affiliatedwith → University of Bologna
confidence 99% · G. Li Pira and F. Starita are with the University of Bologna, Department of Psychology
Jingfeng Shan → affiliatedwith → WiLab
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Autonomous agents operating in uncertain environments must balance fast responses with goal-directed planning. Classical MF RL often converges slowly and may induce unsafe exploration, whereas MB methods are computationally expensive and sensitive to model mismatch. This paper presents a human-inspired hybrid RL architecture integrating Pavlovian, Instrumental MF, and Instrumental MB components. Inspired by Pavlovian and Instrumental learning from neuroscience, the framework considers contextual radio cues, here intended as georeferenced environmental features acting as CS, to shape intrinsic value signals and bias decision-making. Learning is further modulated by internal motivational drives through a dedicated motivational signal. A Bayesian arbitration mechanism adaptively blends MF and MB estimates based on predicted reliability. Simulation results show that the hybrid approach accelerates learning, improves operational safety, and reduces navigation in high-uncertainty regions compared to standard RL baselines. Pavlovian conditioning promotes safer exploration and faster convergence, while arbitration enables a smooth transition from exploration to efficient, plan-driven exploitation. Overall, the results highlight the benefits of biologically inspired modularity for robust and adaptive autonomous systems under uncertainty.
Tags
Links
- Source: https://arxiv.org/abs/2603.22170v1
- Canonical: https://arxiv.org/abs/2603.22170v1
Trouble viewing inline? Open PDF directly →
Full Text
91,804 characters extracted from source content.
Expand or collapse full text
Human-Inspired Pavlovian and Instrumental Learning for Autonomous Agents Navigation Jingfeng Shan, Member, IEEE University of Bologna, Bologna, Italy Francesco Guidi, Member, IEEE National Research Council of Italy, Bologna, Italy Mehrdad Saeidi, Member, IEEE University of Bologna, Bologna, Italy Enrico Testi, Member, IEEE University of Bologna, Bologna, Italy Elia Favarelli, Member, IEEE University of Bologna, Bologna, Italy Andrea Giorgetti, Senior Member, IEEE University of Bologna, Cesena, Italy Davide Dardari, Fellow, IEEE University of Bologna, Cesena, Italy Alberto Zanella, Senior Member, IEEE National Research Council of Italy, Bologna, Italy Giorgio Li Pira University of Bologna, Bologna, Italy Francesca Starita University of Bologna, Bologna, Italy Anna Guerra, Member, IEEE University of Bologna, Cesena, Italy Abstract— Autonomous agents operating in uncertain environ- ments must balance fast responses with goal-directed planning. Classical Model-Free (MF) reinforcement learning (RL) often converges slowly and may induce unsafe exploration, whereas Model-Based (MB) methods are computationally expensive and This work was partially supported by the European Union under the NRPP of NextGenerationEU (Mision 4 – Component 2 -Investment 1.1) Prin 2022 (No. 104, 2/2/2022, CUP J53C24002790006), under ERC Grant no. 101116257 (project CUE-GO: Contextual Radio Cues for Enhancing Decision Making in Networks of Autonomous Agents). (Corresponding author: J, Shan). J. Shan, M. Saeidi, E. Testi, E. Favarelli, D. Dardari, and A. Guerra are with WiLab, University of Bologna, via dell’Universit ́ a 50, 47521 Cesena, Italy, e-mail: enrico.testi, elia.favarelli, andrea.giorgetti, da- vide.dardari, anna.guerra3@unibo.it. F. Guidi and A. Zanella are with the National Research Council of Italy (CNR), viale del Risorgimento 2, 40136 Bologna, Italy, e-mail: francesco.guidi, alberto.zanella@cnr.it. G. Li Pira and F. Starita are with the University of Bologna, Department of Psychology “Renzo Canestrari”, 47521 Cesena, Italy. 0018-9251 © IEEE sensitive to model mismatch. This paper presents a human-inspired hybrid RL architecture integrating Pavlovian, Instrumental MF, and Instrumental MB components. Inspired by Pavlovian and Instrumental learning from neuroscience, the framework considers contextual radio cue, here intended as georeferenced environmental features acting as conditioned stimulus (CS), to shape intrinsic value signals and bias decision-making. Learning is further modulated by internal motivational drives through a dedicated motivational signal. A Bayesian arbitration mechanism adaptively blends MF and MB estimates based on predicted reliability. Simulation results show that the hybrid approach accelerates learning, improves operational safety, and reduces navigation in high-uncertainty regions compared to standard RL baselines. Pavlovian conditioning promotes safer ex- ploration and faster convergence, while arbitration enables a smooth transition from exploration to efficient, plan-driven exploitation. Overall, the results highlight the benefits of biologically inspired modularity for robust and adaptive autonomous systems under uncertainty. Index Terms— Multi-agent Reinforcement Learning, Pavlovian- Instrumental Transfer, Autonomous Agent, and Localization. I. Introduction Autonomous agents such as unmanned aerial vehicles (UAVs), mobile robots, and autonomous ground vehicles are increasingly deployed in mission-critical applications, including post-disaster scenarios [1]. These environments are characterized by high uncertainty, degraded communi- cation infrastructures, and stringent real-time constraints [2]–[4]. Agents must operate on noisy, high-dimensional sensory data, adapt to unpredictable disturbances, and make decisions that balance immediate task demands with long-term objectives. A core challenge in autonomous agent networks is transforming uncertain observations into reliable state estimates and mapping them to control actions that satisfy safety, efficiency, and performance requirements under limited computational and energy resources [5]–[7]. Most autonomous decision-making approaches rely on Model-Free (MF) reinforcement learning (RL), where agents iteratively refine policies through reward feedback and environmental interaction [8]–[11]. While effective in controlled settings, these methods treat learning as a monolithic optimization process driven by uniform temporal-difference (TD) updates. This perspective con- trasts with biological decision-making systems, where multiple functionally distinct learning mechanisms coexist and interact [12]–[14]. Such parallel systems contribute differently to behavioral control, enabling faster learning while balancing safety and efficiency in unfamiliar and potentially hazardous environments. Neuroscientificevidenceindicatesthathuman decision-makingisgovernedbyatleasttwo dissociable learning systems [15]–[17]. Instrumental conditioning supports goal-directed behavior by learning action–outcome contingencies, reinforcing actions that lead to favorable outcomes while suppressing harmful ones [18]. In contrast, Pavlovian conditioning learns stimulus–outcome associations, allowing organisms to anticipate rewards or punishments based on predictive environmental cues, independently of their actions [19]– IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No.1 arXiv:2603.22170v1 [cs.MA] 23 Mar 2026 [21]. Pavlovian learning enables fast, reflexive responses to contextual cues, which in engineered systems mayincluderadio-derivedenvironmentalfeatures, supportinganticipatorybehaviorinsafety-critical scenarios.Importantly,Pavlovianandinstrumental systems operate in parallel, with Pavlovian predictions biasing instrumental action selection toward expected outcomes. Motivated by these insights, the authors in [22] advocated closer integration between neuroscience and robotics, proposing a conceptual hierarchical architecture comprising reflexes, Pavlovian responses, MF habitual control, and Model-Based (MB) goal-directed planning, coordinated by an arbitration mechanism sensitive to uncertainty. However, their contribution remains largely at a theoretical level, without specifying a computational implementation or a concrete learning algorithm for au- tonomous agents. Then, [23] proposed the PAL algorithm that integrates Pavlovian fear signals into standard RL through uncer- tainty, modulated action biases. The MaxPain framework [24] separates reward and punishment learning via dis- tinct Q-functions, yielding more conservative exploration, while its extension in [25] introduces signal-specific dis- counting and adaptive weighting of subsystems. Despite these advances, it is still unclear how Pavlovian stimulus–outcome predictions interact with in- strumental action–outcome learning to shape decision- making. This interaction is named in neuroscience as Pavlovian-Instrumental Transfer (PIT) [26], [27], whereby Pavlovian conditioned stimuli modulate instrumental be- havior. PIT manifests in specific, general, and inhibitory forms, influencing action selection even when cues pro- vide no information about the optimal action. This high- lights a deep interaction between prediction-based and control-based learning mechanisms rather than simple information fusion. From a computational perspective, PIT poses a sig- nificant challenge to standard actor–critic architectures, which conflate state-value estimation and policy opti- mization and therefore fail to capture several key PIT phenomena [13]. In particular, such models struggle to account for reflexive, stimulus-driven responses that occur independently of action–outcome contingencies, for the persistent modulation of behavior by internal motivational states even after extensive training, and for Pavlovian biases that can systematically influence action selection in ways that may be suboptimal from a purely instrumental standpoint. The advantage function, defined as the difference between state-action values and state values, has been proposed as a natural computational substrate for mod- eling PIT effects in computational neuroscience [13], [28], [29]. Recent works have leveraged advantage-based formulations in multi-agent and robotic systems [30], [31] to improve convergence and coordination. Within this framework, the state-value component can be interpreted as a Pavlovian predictor, while action-specific advantages encode instrumental control. As learning progresses and optimal actions are repeatedly selected, advantages dimin- ish, yielding a transition from deliberative control to au- tomatic, value-driven responding, thus mirroring the shift from goal-directed to habitual behavior under Pavlovian influence [13]. Unfortunately, advantage-based PIT models remain largely unexplored in the design of autonomous naviga- tion of agents. As an example, in the context of target localization under GPS-denied navigation, agents must rapidly exploit contextual environmental cues, including radio signal features, to anticipate risks and opportunities. Learning predictive associations between such cues and task-relevant outcomes (e.g., localization accuracy) could substantially enhance robustness and adaptivity. More- over, explicitly separating Pavlovian and instrumental processes may enable heterogeneous multi-agent systems in which agents exhibit complementary behaviors, im- proving collective performance in complex environments. Building on these insights, and on [32] where we provide a first discussion on the human-inspired architec- ture, this work introduces a novel Pavlovian–Instrumental RL architecture, in which PIT mechanisms are explic- itly realized within a MF learning framework, while a complementary hybrid MB/MF component provides goal- directed planning through arbitration, thereby enhancing autonomous decision-making in unknown environments. The main contributions of this paper are summarized as follows. • We propose a human-inspired decision-making framework integrating three learning systems: a Pavlovian module for cue-based modulation, an In- strumental MF learner for habitual control, and an Instrumental MB planner for goal-directed planning. • We introduce the concept of contextual environmen- tal cues, defined as georeferenced features that act as conditioned stimulus (CS) and generate intrin- sic Pavlovian value signals, thereby biasing action selection toward information-rich regions and away from hazardous areas. The proposed framework is general and can accommodate different types of environmental cues. In this work, we focus on radio- based cues as a representative example of practical interest for wireless localization, and adopt them in the numerical results. • We propose a novel learning architecture that in- tegrates Pavlovian and instrumental components. Pavlovian predictions associated with environmental cues are converted, through a dedicated modulation function, into action-dependent approach-avoidance biases that influence the agent’s action selection. A motivational gate further modulates learning dynam- ics based on internal state variables, such as battery level and mission time. • We validate the proposed framework in a multi-agent target localization task, demonstrating improved per- formance compared to standard RL baselines. 2IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. TABLE I: Neuroscience Acronyms AcronymMeaning NSNeutral Stimulus CSConditioned Stimulus USUnconditioned Stimulus URUnconditioned Response CRConditioned Response PITPavlovian-to-Instrumental Transfer The remainder of this paper is organized as follows. Section I defines the problem formulation, reviews the biological background of instrumental and Pavlovian con- ditioning, and presents the proposed architecture. Sections I describes the considered framework, while Sections IV reports the proposed algorithm and the hybrid approach. Section V describes simulation results, while final con- clusions are drawn in Section VI. Table I summarizes the main acronyms from behavioral neuroscience used throughout the Pavlovian framework. I. Human-Inspired Digital Agent Architecture A. Research Problem Autonomous agents deployed in complex environ- ments must meet several, often conflicting, requirements: efficient exploration, safe navigation, rapid adaptation to uncertain sensory conditions, and effective long-horizon reasoning. Traditional MF RL methods often struggle in such settings, as they rely solely on reward-driven trial- and-error and lack mechanisms for leveraging predictive environmental cues or expressing innate behavioral bi- ases. Neuroscientific evidence demonstrates that biological decision-making is governed by the interaction of multiple dissociable learning systems, including Pavlovian, MF, and MB controllers, which contribute unique strengths spanning rapid reactivity, robustness to uncertainty, and long-term foresight [13], [22]. Yet, despite substantial work on biologically inspired RL [12], a principled in- tegration of these subsystems remains underexplored in autonomous agents, especially in scenarios where contex- tual cues (e.g., radio features) carry task-related meaning. This paper addresses the following research question: How can Pavlovian, instrumental MF, and MB learn- ing be integrated into a unified decision architecture that enables autonomous agents to navigate and localize targets efficiently in unknown environments? To tackle this problem, we develop a human-inspired hybrid RL algorithm that: (i) incorporates Pavlovian value signals derived from contextual radio cues, (i) maintains instrumental value estimates through both MF and MB updates, and (i) dynamically arbitrates between MF and MB control using a Bayesian reliability mechanism. The proposed framework is evaluated in a multi-agent localization task featuring gates, obstacles, and GPS- denied regions, where anticipatory cue-guided behavior is essential for safe and efficient mission performance. B. Background on Instrumental and Pavlovian Conditioning Pavlovian and instrumental conditioning constitute two distinct learning mechanisms with complementary computational roles [13], [33]. Pavlovian condition- ing provides fast, reflexive responses driven by stim- ulus–outcome associations, whereas instrumental condi- tioning supports flexible action–outcome learning and pol- icy optimization. Their interaction is central to designing agents that combine rapid, cue-driven behavior with goal- directed control. Pavlovian Conditioning In animals, Pavlovian conditioning involves learn- ing predictive relationships between environmental cues and biologically significant outcomes. Through repeated pairings between a neutral stimulus (NS) and an unconditioned stimulus (US), (i.e. a biologically relevant stimulus such as food or pain), the NS becomes a CS capable of triggering a conditioned response (CR). A classic example is Pavlov’s experiment, where a neutral auditory cue paired with food became sufficient to elicit salivation. Pavlovian responses are automatic and rely on evo- lutionarily specified stimulus–response mappings rather than action selection. According to [13], these responses reflect both (i) a hard-wired motivational system, immedi- ately sensitive to internal states (e.g., hunger, thirst), and (i) a learned component mediated by CS–US associa- tions. Instrumental Conditioning Instrumental conditioning concerns learning ac- tion–outcome relationships. It relies on two systems: (i) Habitual (MF): acquires stimulus–response associa- tions via reinforcement and often becomes insensitive to outcome revaluation following extensive experience with the association (so called, overtraining); (i) Goal- directed (MB): selects actions by evaluating predicted consequences and is influenced by the agent’s internal motivational state, ensuring that instrumental actions are inhibited when the predicted outcome no longer carries incentive value (e.g., the absence of hunger suppresses the pursuit of food) [13]. Pavlovian–Instrumental Transfer (PIT) Pavlovian cues can energize or bias instrumental ac- tions, a phenomenon known as PIT [26]. The authors in [13] highlighted that PIT is driven by the Pavlovian motivational system and is neurally dissociable from goal- directed incentive learning. Consequently, Pavlovian value can modulate instrumental policies even in the absence of direct action–outcome information. This mechanism motivates the hybrid control scheme adopted in this work, where Pavlovian value modulates advantage-based instru- mental decision making, enabling both rapid reactivity and long-horizon planning. Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.3 C. Human-Inspired Learning Architecture Fig. 1 illustrates the proposed human-inspired hybrid learning architecture, which governs how an autonomous agent interacts with its environment and internally regu- lates perception, learning, and action selection. The design is conceptually inspired by biological decision-making frameworks such as [22], in which multiple learning systems operate in parallel and are dynamically combined based on their reliability and contextual demands. Unlike the biological framework in [22], which fo- cuses on modeling neural decision mechanisms, the proposed architecture translates these principles into a computational control framework for autonomous dig- ital agents. In particular, it introduces (i) contextual environmental cues derived from radio features, (i) a Pavlovian mechanism that generates action-dependent ap- proach–avoidance biases, and (i) a motivational gating mechanism that modulates learning according to opera- tional constraints such as battery level and mission time. 1. External Perception–Action Loop The perception–action loop constitutes the interaction between the agent and the external environment. At each timestep, the agent acquires sensory inputs from onboard sensors (e.g., radio measurements, cameras), which en- code partial and noisy information about the environment. These observations are processed through learning and decision processes. Actions generated by the decision module alter the environment and yield new sensory inputs or external reinforcement signals, thereby closing the loop. This mechanism corresponds to the top-level flow in Fig. 1 (dashed lines), where perception and action continuously interact to shape the agent’s experience. 2. Internal Environment Although sensory information originates externally, its interpretation is fundamentally shaped by the organism’s internal state. In biological systems, the brain does not assign fixed meaning to environmental stimuli; rather, it continuously evaluates them against ongoing physiologi- cal needs, so that the same cue can carry vastly different motivational weight depending on the current internal condition [34]. This process is anchored in interoception, the neural mapping of bodily signals such as energy depletion, which provides the foundational input for all subsequent value computation and behavioral prioritiza- tion [35]. Crucially, motivation itself is not a single signal but emerges from the interplay of two distinct processes: the encoding of internal need states, which sets the overall level of behavioral urgency, and the affective weighting of external stimuli, which determines which actions are pursued to satisfy those needs. These two processes are mirrored in the central block of Fig. 1 through two cooperating components: • Motivational Gate: evaluates internal variables such as battery level and mission time (namely, inter- nal state). As resources deviate from their optimal operating range, the gate generates a motivational signal whose magnitude reflects the degree of inter- nal deficit. This signal modulates reward sensitivity by introducing a state-dependent cost term, effec- tively implementing a closed-loop negative feedback mechanism. Increased resource depletion elevates the penalty term, suppressing inefficient behaviors and biasing the agent toward resource-efficient tra- jectories. As behavior becomes more efficient and resource consumption stabilizes, the motivational signal correspondingly decreases, thereby complet- ing the regulatory loop. A mathematical formulation is provided in Sec. C. • Affective Representation: assigns motivational sig- nificance to sensory inputs, enabling the agent to interpret specific environmental conditions as inter- nally rewarding or punishing depending on the exe- cuted action. While the “Motivational Gate” encodes urgency, the “Affective Representation” determines directional value assignment to external cues. The reward model is detailed in Sec. E. Together, these elements generate intrinsic reward-like signals that influence both MF and MB learning sys- tems. Importantly, because the motivational modulation operates through bounded additive cost adjustment rather than multiplicative reward amplification, the resulting temporal-difference errors remain stable. This ensures that motivational urgency regulates behavior without desta- bilizing value learning, mirroring biological homeostatic control systems in which internal deficits dynamically constrain action selection. 3. PIT Model-Free Learning The PIT MF module (orange block in Fig. 1) inte- grates two subsystems: Pavlovian Module Learns predictive associations between contextual cues and motivationally significant outcomes, following approaches as in [36]. Its output is a state-dependent Pavlovian value that biases action selection toward ap- proach or avoidance tendencies and/or action invigoration or suppression. The learning mechanism is described in Sec. D. Instrumental Model-Free RL Learns action-outcome associations through experi- ence, producing habitual action values that guide fast, reactive behavior. The instrumental learning is described in Sec. D. This subsystem generates a reward prediction error (RPE) used to assess reliability for arbitration, described in Sec. D. 4. Model-Based Learning The MB learner (yellow block in Fig. 1) constructs and updates an internal transition model derived from experience. Using this model, the agent performs planning via simulated rollouts (Dyna-Q), enabling planning and 4IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. External Environment Sensory Input State/ObservationReward/Penalty Action Agent Policy Design & Action Selection with Pavlovian Bias Motivational Signal Instrumental RL Q-Value Update RPE Computation PIT Model-Free Learning Forward Learner Dyna-Q Planning SPE Computation Experience Memory Model Model-Based Learning Internal Environment Other Sensory Representations Affective Representation Motivational Gate Dynamic Probability Calculation Hybrid Q-Value Calculation SPE RPE Model-Free Q-ValueModel-Based Q-Value Pavlovian Value Arbitration Module Pavlovian Bias TD Value Update Internal State Fig. 1: Introduced hybrid learning architecture combining MB and MF processes, inspired by neuroscience [22]. long-term reasoning, and computes the corresponding model-based Q-values used for action evaluation. The MB update rules and planning cycle are detailed in Sec. IV. This subsystem also computes a state prediction error (SPE) used for arbitration as described in Sec. D. 5. Arbitration Mechanism The Bayesian arbitration mechanism (green block in Fig. 1) evaluates the relative reliability of the MF and MB subsystems based on their respective prediction errors. The resulting arbitration probability determines the contribution of each subsystem to the final value estimate. The arbitration process is formally described in Sec. D. 6. Policy Design and Action Selection The final decision-making stage combines MF and MB value estimates into a hybrid action-value repre- sentation, weighted by the arbitration probability, and biased by Pavlovian values. The resulting logits drive a softmax policy that selects the agent’s action. The policy evaluation and action selection mechanism is detailed in Secs. F and D. Overall, this architecture enables the agent to inte- grate cue-driven responses, habitual control, and planning, achieving robust decision-making in uncertain and com- plex environments. I. Pavlovian–Instrumental Transfer (PIT) Model-Free (MF) RL Framework We now introduce the proposed hybrid architecture incorporating PIT for target localization, illustrated in Fig. 1. A. Environment and Agent State Representation We consider N autonomous agents operating in an unknown environment. Their objective is to navigate the area and search for a hidden target located at p T by guaranteeing a localization error, in terms of Position Error Bound (PEB), below a desired threshold. The environment is modeled as a discrete 2D grid with cell size ∆, where each cell may be free, occupied by walls, agents, or the target, or correspond to contextual cues. Each agent learns independently and maintains its own state–action value function through an individual Q- table, i.e., a tabular representation that stores the estimated value Q(s,a) associated with taking action a in state s. This design follows the independent learners paradigm commonly adopted in multi-agent reinforcement learning, where each agent updates its policy based on local ex- perience while interacting with other agents through the shared environment. As a result, coordination may emerge implicitly through the environment dynamics rather than through explicit information sharing. 1 The state of agent i at time t is given by its current position, s i,t = p i,t ∈S with i ∈ N = 1, 2,...,N. Here, S denotes the state space, i.e., the set of all possible positions an agent can occupy. At each step, the agent selects an action a i,t ∈ A = up, down, right, left, hover, each repre- sented as a displacement vector of magnitude ∆ (e.g., left = [−∆, 0]). In addition to its external state, each agent maintains an internal state vectorι i,t = [b i,t , τ i,t ] T , where b i,t denotes the remaining battery level and τ i,t the elapsed mission time. Executing the hover action consumes less energy than movement actions, reflecting its lower op- erational cost. Agents are assumed to be equipped with RF sensing devices that allow them to observe electro- magnetic features of the environment and exploit them as contextual cues for decision making. B. Contextual Radio Cues As previously discussed, potential cues are initially NS that become, after association, predictive of an out- come. In this regard, let here define G = G gate ∪G GD = 1,...,g,...G as the set of indexes referring to en- vironmental states representing potential cues. Cues are here divided into two categories: (i) gates (g ∈ G gate ), namely features extracted from the electromagnetic (EM) map corresponding to openings (doors or windows) 2 that may indicate a transition from non-line-of-sight (NLOS) to line-of-sight (LOS) propagation conditions; (i) GPS- denied regions (g ∈ G GD ), where agents lose access to absolute positioning, leading to increased self-localization 1 It is worth noting that, under the MB approach, each agent maintains and updates an independent Q-table alongside its MF counterpart. 2 In this work, we assume that a preliminary phase that comprises the reconstruction of an electromagnetic map of the environment and the extraction of semantic radio features (e.g., the shape of a door) has already been completed. Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.5 uncertainty, which in turn degrades target-localization performance. Initially perceived as NS, these potential cues become CS through repeated association with favorable outcomes, with the target state functioning as the US. 3 Table I summarizes the parallelism between Pavlovian learning in neuroscience for humans and our interpretation in this paper for autonomous agents. For instance, only those gates that reliably lead to advantageous areas (e.g., LOS regions facilitating target discovery) are validated as CSs, whereas gates that do not improve target localization remain behaviorally irrelevant. In the remainder of this work, we assume that the association between CS and out- come has already been established. A possible algorithm to implement such cue–outcome association is discussed in [36]. C. Motivational Gate and Internal Drive Modulation To mimic how internal motivational drives influence biological behavior, we define a motivational signal M i,t for autonomous agents as M i,t = ξ 1 (1− ̄ b i,t ) + ξ 2 · ̄τ i,t ,(1) where ̄ b i,t = b i,t B max ∈ [0, 1], ̄τ i,t = τ i,t T max ∈ [0, 1] denote the normalized battery level and elapsed mission time, respectively. B max and T max refer to the maximum battery level and available mission time, while ξ 1 and ξ 2 represent the weights for the battery and mission time components, respectively. In this work, these parameters are treated as design parameters and selected empirically, the values used in the experiments are reported in Ta- ble I. This motivational mechanism is inspired by biologi- cal principles in humans, where motivation arises from metabolic needs (e.g., hunger or thirst), emotional states, and goal-directed behaviors. At the neural level, intero- ceptive signals, originating from visceral, metabolic, and somatic receptors, are continuously mapped by the insular cortex into structured representations of physiological state, providing the foundational input for all subsequent motivational and value computations [35]. These signals are relayed along a posterior–anterior gradient through brainstem nuclei, thalamus, and insula, before reaching higher-order regions such as the anterior cingulate cortex (ACC) and ventromedial prefrontal cortex (vmPFC) [37], where they are progressively integrated with affective, motivational, and cognitive information. Homeostatic deficits, such as low energy reserves or time pressure, are translated into directed motivational drives primarily via the lateral hypothalamus (LH), which 3 Throughout this work, the terms potential cue and neutral stimulus are used interchangeably; similarly, validated cue and CS are considered equivalent. encodes motivational salience in a state-dependent man- ner, sharpening the distinction between appetitive and aversive inputs as need states intensify [38], [39]. Full motivational activation emerges through the mesolimbic dopamine system, especially the projections from the ventral tegmental area (VTA) to the nucleus accumbens, which attribute incentive salience, or “wanting,” to cues and actions relevant to current needs [40]. Crucially, this cue-dependent mechanism is complemented by a hypotha- lamus–ventral striatum–VTA seeking circuit capable of independently energizing exploratory behavior even in the absence of explicit external cues, suggesting that internal need states alone carry sufficient motivational weight to initiate action [41]. Beyond energizing behavior, these biological systems operate as regulatory feedback loops: as physiological resources deviate from their optimal range, deficit signals intensify, increasing motivational urgency and biasing behavior toward resource-restoring actions [42]. Once homeostasis is re-established, the urgency signal dimin- ishes, stabilizing behavior. Motivation thus functions not merely as amplification, but as dynamic negative feedback control over action selection and value computation. Drawing on this analogy, the motivational signal in (1) allows each autonomous agent to adjust its learning dynamics and decision-making according to its internal state, thus capturing a simplified affect-like modulation [13], [43]. The two components of M i,t correspond to distinct yet complementary biological drives. The battery depletion term ̄ b i,t = b i,t B max ∈ [0, 1] reflects interoceptive deficit signals, similar to hunger or energetic need, that increase motivational urgency as resources decline, while the elapsed mission time term ̄τ i,t = τ i,t T max ∈ [0, 1] captures a goal-directed temporal pressure, analogous to the increasing urgency to find food the longer an organism has gone without eating. Within the proposed architecture, these signals mod- ulate value computation through an additive cost term, implementing a closed-loop negative feedback mechanism [42]. As internal deficits increase, the motivational penalty strengthens, suppressing inefficient or resource-intensive behaviors. The resulting policy adjustment reduces further depletion, which in turn attenuates the motivational signal. Through this recursive interaction, the agent’s behavior becomes progressively more resource-efficient under con- straint, while maintaining stable learning dynamics. D. Pavlovian and Instrumental Learning In our framework, both the instrumental and Pavlovian systems rely on value-based RL. Formally, following the standard definition by [33], the value of a generic state s t = s under policy π is defined as V π (s) = E π " ∞ X k=0 γ k r t+k+1 |s t = s # ,(2) where s t denotes the state observed by a generic agent at time t. For notational simplicity, the agent index i used 6IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. TABLE I: Parallelism between Pavlovian conditioning, and our proposed framework. Pavlovian learning (neuroscience)DescriptionIn this paper Neutral Stimulus (NS)State with no intrinsic value, that is, before asso- ciation they are not linked with task outcomes. (i) GPS-denied regions (i) Gates These contextual radio features, before associa- tion, are not initially associated with task out- comes. Unconditioned Stimulus (US)Task-relevant outcome signal.Target localization metrics, such as the position error bound (PEB), depend on various factors. Among them, we focus on: (i) Agents’ position accuracy, (i) LOS propagation conditions. Conditioned Stimulus (CS)Predictive state acquired through learning.When GPS-denied regions and gates are consid- ered validated contextual radio-cues predictive of outcomes learned from association: (i) GPS-denied regions anticipate uncertainty on agents’ positions and, thus, decreased PEB (i) Gates anticipate LOS propagation and, thus, improved PEB Unconditioned Response (UR)Immediate reflexive response to reinforcement.Instantaneous value update driven by reward-based temporal-difference error. Conditioned Response (CR)Automatic, stimulus-driven behavioral bias.(i) GPS-denied region: avoidance tendency in- duced by Pavlovian values through action- dependent policy modulation. (i) Gates: approach tendency induced by Pavlo- vian values through action-dependent policy mod- ulation. elsewhere in the paper is omitted in this definition. r t+1 is the reward received after transitioning from s t to the future state s t+1 , γ is the discount factor (0 ≤ γ ≤ 1), determining the weight of future rewards relative to immediate ones. Similarly, the action-value function is expressed as Q π (s,a) = E π " ∞ X k=0 γ k r t+k+1 |s t = s,a t = a # .(3) Note that the expectation in (2)-(3) is taken over the stochastic trajectories induced by the environment dy- namics and by the policy itself. Building on these defini- tions, our proposed framework in Fig. 1 decomposes RL into two interacting systems: an instrumental system that learns action-outcome contingencies through trial-and- error, and a Pavlovian system that predicts motivationally significant outcomes based on contextual radio cues. The instrumental MF system employs off-policy TD learning (Q-learning) to estimate the state–action value function Q MF,i (s i,t , a i,t ), ∀i ∈ N . More specifically, for the i-th agent, the state-action value is updated as Q MF,i (s i,t , a i,t )← Q MF,i (s i,t , a i,t ) + α Q δ Q,i,t ,(4) with α Q being the learning rate and δ Q,i,t being the TD error computed as δ Q,i,t =(r i,t+1 − φM i,t ) + γ Q max a Q MF,i (s i,t+1 , a i,t+1 )− Q MF,i (s i,t , a i,t ), (5) where φ is a weight to balance the strength of motivational signal. In parallel, the Pavlovian module learns a value func- tion that for the i-th agent is given by V i (s i,t )← V i (s i,t ) + α V δ V,i,t ,(6) with α V being the learning rate and δ V,i,t being the TD error computed as δ V,i,t = (r i,t+1 − φM i,t ) + γ V V i (s i,t+1 )− V i (s i,t ). (7) In our architecture, the Pavlovian learner maps con- textual radio cues into state-dependent value estimates V i (s i,t ), capturing the motivational significance value in- stead of motivational significance of each cue indepen- dently of the available actions. In this formulation, the Pavlovian component provides a cue-driven valuation of the state, while instrumental learning retains responsibility for discriminating among actions. As learning progresses, instrumental action pref- erences become more stable, and behavior is increasingly shaped by the Pavlovian value associated with the current context, in line with established theoretical accounts of PIT interactions [13]. This mechanism parallels biological motivational sys- tems, where subcortical structures such as the hypothala- mus, amygdala, and ventral tegmental area assign value to sensory cues and modulate action readiness prior to instrumental deliberation [34]. Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.7 E. Reward Model In this subsection, we describe the reward structure used for each agent. 4 The instantaneous reward received by agent i after executing action a i,t in state s i,t is defined as r i,t+1 (s i,t , a i,t ) = r inst,i,t+1 + r pav,i,t+1 ,(8) where r inst,i,t+1 and r pav,i,t+1 denote the instrumental and Pavlovian components, respectively, as detailed next. Instrumental Conditioning Reward The instrumental reward consists of two components: (i) an RSSI-based shaped reward encouraging movement toward positions with higher detection and localization quality, and (i) a risk penalty discouraging collisions with walls, agents, or the target. Formally, r inst,i,t+1 (s i,t , a i,t ) = r RSSI,i,t+1 + r risk,i,t+1 ,(9) where r RSSI,i,t+1 is the reward derived from the received signal strength, and r risk,i,t+1 penalizes dangerous actions. We shape the RSSI-based reward using the measured signal strength RSS i,t at each time step. Since higher RSSI (and, thus, higher SNR) increases the likelihood of target detection, we adopt a normalized reward defined as r RSSI,i,t+1 = ̄ P r,i,t (10) where ̄ P r,i,t = P r,i,t /P r,max is the received power normal- ized with respect to its maximum possible value. Notably, the received power (in [dBm]) at agent i can computed via the link-budget model: P r,i,t [dBm] = P t [dBm]+G r,i [dBi]+G t [dBi]−L i,t [dB]+S h , (11) where P t is the transmit power, G t and G r,i are antenna gains, S h ∼N (0,σ 2 s ) models log-normal shadowing, and L i,t = PL i,t +L walls,i,t accounts for propagation and wall losses. The path-loss term is PL i,t [dB] = 20 log 10 4π λ cw + 10η log 10 (d i,t ),(12) with d i,t the distance from agent to target, η the varied path-loss exponent depending on LOS or NLOS, 5 and λ cw the carrier wavelength. The reference maximum received power P r,max [dBm] in (10) is computed assuming minimum path-loss at d 0 = 1 m (with grid cell size ∆ = 1 m). The risk penalty reflects collision events. LetO be the set of cells occupied by obstacles or other agents at time t + 1. Then, r risk,i,t+1 =−λ col 1collision i,t+1 ,(13) with 1collision i,t+1 = 1 if p i,t+1 ∈O, 1 if ∥p i,t+1 − p T ∥≤ d safe , 0 otherwise, (14) 4 Throughout this work, the term reward refers to both positive rewards and negative rewards (i.e., penalties). 5 Since the target periodically transmits a beacon, the path-loss corre- sponds to a one-way link from target to agent. and λ col and d safe are the collision penalty, and distance threshold between agents and the target, respectively. The operator 1 denotes the indicator function, which is used to distinguish between cases. Pavlovian Reward The Pavlovian reward captures both positive and neg- ative reinforcements arising from contextual radio cues, and is defined as r pav,i,t+1 (s i,t , a i,t ) = r gate,i,t+1 + r GD,i,t+1 .(15) Here, r gate,i,t+1 provides a positive reinforcement when the next state belongs to a gate region (as gates are as- sociated with NLOS–LOS transitions, where approaching from NLOS typically improves propagation conditions), whereas r GD,i,t+1 introduces a negative penalty when entering a GPS-denied cell. Formally, r gate,i,t+1 = ( +|r|, if s i,t ∈S NLOS , s i,t+1 ∈G gate , 0,otherwise, (16) r GD,i,t+1 = ( −|r|, s i,t+1 ∈G GD , 0,otherwise, (17) where r is a fixed constant, and S NLOS denotes the set of NLOS states. Goal Reward A terminal reward R goal is assigned to each agent upon successful mission completion. In our RL formulation, an episode corresponds to a complete mission execution, starting from the initial deployment of the agents and ending when a termination condition or maximum mis- sion time is met. The mission is considered accomplished when the target localization accuracy reaches a prescribed threshold. Localization accuracy is quantified through the PEB, and an episode terminates whenever PEB≤ PEB ∗ . The PEB is defined as the square root of the trace of the Cram ́ er-Rao Lower Bound (CRLB) matrix, namely PEB t = p trace(CRLB t ) = q trace(J −1 t ),(18) where J is the Fisher Information Matrix (FIM) for the target position. 6 Assuming that each agent acquires ranging measure- ments, the FIM is given by J t = N X i=1 J i,t = N X i=1 1 σ 2 i,t u i,t u ⊤ i,t ∈ R 2×2 ,(19) where u i,t is the unit vector from agent i to the target, expressed as u i,t = p T − ̃ p i,t ∥p T − ̃ p i,t ∥ ∈ R 2×1 ,(20) 6 The CRLB expression follows from standard regularity conditions on the likelihood function-differentiability with respect to p T , integrability of the score function, and vanishing boundary terms. In our simulations, if the FIM J is not well-conditioned, we set PEB→ +∞. 8IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. and ̃ p i,t = p i,t +N (0, Σ GPS,i,t ) denotes the estimated agent position affected by GPS error with covariance matrix Σ GPS,i,t . The total measurement variance σ 2 i,t appearing in (19) is σ 2 i,t = σ 2 r,i,t + u ⊤ i,t Σ GPS,i,t u i,t ,(21) where σ 2 r,i,t denotes the variance of the ranging mea- surement, Σ GPS,i,t is the covariance matrix of the agent position estimate. Moreover, since the GPS error affects the effective range measurement through the uncertainty in the agent position, the second term accounts for the pro- jection of the position uncertainty onto the agent–target direction. The ranging variance is approximated as [44] 7 σ 2 r,i,t ≈ c 2 8π 2 SNR i,t β 2 eff ,(22) with c the speed of light, β 2 eff the effective bandwidth, and SNR i,t the linear SNR at time t. The SNR is computed as SNR i,t = P r,i,t /P n,i where P n,i is the receiver noise power at the ith agent. F. Policy Evaluation with Pavlovian Modulation A large body of computational work has examined how Pavlovian influences modulate instrumental action selection. Early models, such as [45], considered decision problems in which agents choose between executing an action (“Go”) or withholding it (“No-Go”). In this con- text, a Pavlovian “Go bias” was introduced to model an intrinsic tendency toward action invigoration, independent of instrumental action–outcome contingencies. Action se- lection was typically implemented via a softmax rule augmented with an additive term encoding either a general action bias or a cue-dependent tendency to act. Subse- quent studies [46] extended this framework by incorpo- rating Pavlovian value predictions into the same softmax structure. These works highlighted a critical property: if the Pavlovian term enters identically for all actions, it cancels out during softmax normalization, producing no behavioral effect. A genuine Pavlovian influence on instrumental choice emerges only when the Pavlovian component is made action-dependent, for example by selectively enhancing “Go” actions or suppressing “No- Go” actions. More recent models have emphasized the dynamic nature of this interaction. For instance, [47] showed that Pavlovian invigoration can be amplified or inhibited de- pending on neural markers of conflict, such as midfrontal theta power, allowing moment-to-moment modulation of cue-based biases. Other work has focused on arbitration mechanisms between Pavlovian and instrumental con- trollers [48]. These studies propose that Pavlovian biases dominate when outcomes are relatively uncontrollable, 7 The expression in (22) already converts temporal uncertainty into spatial uncertainty through the factor c 2 . As a result, σ 2 ToA,i,t represents a distance–variance term expressed in m 2 . whereas instrumental action values prevail when actions reliably influence rewards. This line of research sup- ports the idea that Pavlovian and instrumental processes contribute jointly to decision-making, with their relative weight determined by environmental structure and uncer- tainty. Motivated by this background, we define the action selection following the conventional softmax policy as π(a i,t | s i,t ) = exp(ω(s i,t , a i,t ))/κ i,t P a∈A exp(ω(s i,t , a))/κ i,t (23) where κ i,t is the temperature parameter controlling the exploration level of the softmax (Boltzmann) policy and it is modulated by modulation gate and defined as κ i,t = κ 0 1+M i,t . The function ω(s i,t , a i,t ) is the action-selection score used by the softmax policy and defined as ω(s i,t , a i,t ) = Q MF,i (s i,t , a i,t ) + β· g a (V i (s i,t ), s i,t , a i,t ), (24) with β being a fixed balancing parameter controlling the trade-off between instrumental and Pavlovian learning. In classical softmax action selection, any term that is identical across actions cancels out during normalization, thereby exerting no influence on the policy. For this rea- son, the Pavlovian contribution must be action-dependent in order to bias choice behavior. In (24), this is achieved through the modulation function g a (·), which assigns different values to each action based on the predicted Pavlovian value of the next state, and it is defined as g a (V i (s i,t ), s i,t , a i,t ) = +|V i (s i,t )|, if s i,t ∈S NLOS , s i,t+1 ∈G gate , −|V i (s i,t )|, if (s i,t , a i,t )→ s i,t+1 ∈G GD , 0,otherwise, (25) where a positive modulation +|V i | promotes approach behavior, while a negative modulation −|V i | implements avoidance of low-value or risky regions. This mechanism allows Pavlovian cue-based predic- tions to bias the softmax policy selectively, enhancing approach tendencies toward desirable regions and discour- aging movement toward unsafe or undesirable zones. Accordingly, the resulting policy in (23) implements a structured deviation from purely value-maximizing be- havior: actions leading toward states with high Pavlo- vian value (e.g., gates) receive an additive positive shift, whereas actions leading toward aversive states (e.g., the GPS-denied area) are penalized by a negative shift. Importantly, this additive modulation preserves the instrumental contingencies encoded in Q i (s i,t , a i,t ) while enabling stimulus-driven influences on the action policy. As a result, the agent exhibits behavior analogous to PIT, in which environmental cues systematically bias action selection even when the cue provides no information about which action is instrumentally optimal. This formu- lation enables autonomous agents to treat radio cues as predictive signals, supporting anticipatory navigation and faster adaptation in environments where specific regions reliably indicate risk or opportunity. Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.9 Algorithm 1: Training Loop for Target Search with PIT MF RL. 1 for ep = 1 : N episodes do 2Reset environment, agent states, and internal variables; 3done ← false; 4for each agent i = 1 : N do 5Initialize agents state s i,1 and internal stateι i,1 ; 6Select initial action a i,1 using softmax policy π(a|s i,1 ) using (9); 7end 8for t = 1 : N steps do 9if done then 10break ;// end episode 11end // Environment transition and measurements 12for each agent i = 1 : N do 13Apply action a i,t and compute next state s i,t+1 (positions, collisions); 14Compute received power, and Fisher Information Matrix using (16) and (25); 15end 16Compute global PEB t .; // Reward and TD updates 17for each agent i = 1 : N do 18Compute motivational signal M i,t using (1); 19Compute instrumental reward r inst,i,t+1 using (13); 20Compute Pavlovian reward r pav,i,t+1 using (20); 21Compute total reward r i,t+1 using (12); 22if PEB t ≤ PEB ∗ then 23r i,t+1 ← r i,t+1 + R goal ; 24done ← true; 25end 26Update Pavlovian value V i (s i,t ) using (6); 27Update instrumental value Q i (s i,t , a i,t ) using (4); 28Select next action a i,t+1 using softmax policy π(a|s i,t+1 ) using (9); 29end 30Update agents state s i,t+1 and internal stateι i,t+1 ; 31end 32 end The complete Pavlovian-Instrumental RL loop, inte- grating the reward structure and the learning rules from the previous subsections, is summarized in Algorithm 1. At every timestep, the agent: (i) executes an action drawn from the softmax policy; (i) observes the next state and evaluates instrumental and Pavlovian rewards; (i) com- putes the motivational signal; and (iv) updates both the instrumental Q-learning value function and the Pavlovian state-value function through their respective TD errors, from which the advantage function is derived to facilitate PIT by decomposing action-values into state baselines and action-specific deviations. The process continues until the mission is terminated in accordance with a localization accuracy criterion. IV. Hybrid Model-Based and Model-Free Learning While the MF system governs habitual, reflexive be- haviors, the MB system enables agents to reason about future states through an internal model of the environment [33]. Thus, differently from MF, MB learning allows agents to evaluate potential actions offline, reducing real- world trial-and-error and enhancing efficiency in uncertain settings. In our framework, in the MB architecture, we employ a Dyna-Q approach, which integrates real experiences with planning to accelerate learning. Following a standard definition and notation [33], each agent maintains a tab- ular internal model M i that records observed transitions and rewards for state-action pairs M i (s,a) = (s ′ ,r,t),(26) where s, s ′ are the current state and the observed next state after executing action a, respectively, r is the received reward, and timestep t records when this transition was last observed. A. Experience Memory Model During real-world interactions at each timestep t, agent i at state/position s i,t executes action a i,t and observes the resulting position s i,t+1 and reward r i,t+1 . The model in (26) is updated as M i (s i,t , a i,t )← (s i,t+1 ,r i,t+1 ,t).(27) This assumes deterministic transitions in our grid world environment. Additionally, the agent maintains a list of visited state-action pairs V i = (s, a)∈S×A (s, a) visited by agent i , (28) = (s i,t k , a i,t k ) L k=1 ,(29) where t k L k=1 denotes the ordered set of time indices at which agent i visited a new state–action pair during the current episode, with L representing the total number of unique state–action pairs encountered. This list is used during planning to sample previously experienced transitions. A state-action pair is added to V i only if it has not been visited before in the current episode, ensuring diverse sampling during planning. 10IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. B. Model-Based Planning with Dyna-Q Architecture After each actual environment transition, that is, after the agent i at state s i,t executes an action a i,t and observes the resulting next state s i,t+1 and reward r i,t+1 , the agent performs K independent planning steps using the Dyna- Q architecture. In this way, these planning steps enable offline value propagation, reducing the need for real- world trials. More specifically, for each planning step k ∈1,...,K, the following phases occur: 1) Random Sampling: State-action pairs (s, a) are sampled uniformly at random from the set of visited pairs V i to efficiently utilize the limited planning budget while ensuring diverse replay of past experiences and avoiding over-concentration on recent transitions: (s i,t , a i,t )∼ Uniform(V i ).(30) This promotes broad exploration of experi- enced transitions without repetition in the current episode. 2) Model Query: Retrieve the stored next state and reward from the internal model ( ˆ s i,t+1 , ˆr i,t+1 ) =M i (s i,t , a i,t ),(31) where ˆ s i,t+1 and ˆr i,t+1 are the next state and reward according to the model. If no transition is stored (i.e., the model entry is empty at early episodes), this planning step is skipped. 3) Temporal Difference Update: Update the model- based Q-table, i.e., Q MB,i (s i,t , a i,t ), using (5) and ˆr i,t+1 . C. Integrated Action Selection and Policy Execution To dynamically arbitrate between MB and MF sys- tems, we propose a hybrid approach that synthesizes the outputs of the MB and MF learning systems under the guidance of a Bayesian arbitration mechanism. The primary input for action selection is a hybrid action-value function, denoted with Q hybrid,i (s i,t , a i,t ), which serves as a unified estimator of the long-term utility of taking action a i,t in state s i,t for agent i and is given by Q hybrid,i (s i,t , a i,t ) = (1− P MB,i,t )· Q MF,i (s i,t , a i,t ) + P MB,i,t · Q MB,i (s i,t , a i,t ),(32) where Q MB,i (s i,t , a i,t ) is the MB Q-table computed using the Dyna-Q architecture described in Sec.B, whereas the Q MF,i (s i,t , a i,t ) is the MF Q-table computed as in (4), and P MB,i,t ∈ [0, 1] is a dynamically updated Bayesian arbitration probability (see also D). The use of a linear combination weighted by a prob- ability ensures the agent’s decision is robust: in states where the environment is predictable and the internal model is accurate (P MB,i,t → 1), the agent relies heavily on foresight and planning. In novel or highly stochastic states where the model fails (P MB,i,t → 0), the agent reverts to proven, habitual responses encoded in the MF system. This adaptive weighting is a key feature enabling the architecture to operate robustly across diverse or changing environmental conditions. Actions are then selected using a softmax policy over the hybrid Q-values, modulated by Pavlovian values. The policy equation in (23) is thus augmented using ω(s i,t , a i,t ) =Q hybrid,i (s i,t , a i,t )+β·g a (V i (s i,t ), s i,t , a i,t ). (33) D. Bayesian Arbitration Mechanism for Model Selection We here derive the arbitration probability P MB,i,t used in (32). Arbitration Probability Following the Bayesian framework proposed in [49], [50] for humans, the arbitration probability for au- tonomous agents can be defined as P MB,i,t = χ MB,i,t χ MB,i,t + χ MF,i,t + ε ,(34) where χ MB,i,t and χ MF,i,t denote the reliability scores of the MB and MF systems (evaluated in (40)), respectively, and ε is a small regularization parameter to prevent division by zero. This quantity satisfies P MB,i,t ∈ [0, 1] and represents the relative confidence placed in the MB system: when χ MB,i,t ≫ χ MF,i,t , we obtain P MB,i,t → 1, giving dominance to MB planning; conversely, when the MF system is more reliable, P MB,i,t → 0, favoring MF- driven control. The arbitration probability is re-computed at every time step, enabling fast adaptation to changes in model accuracy or environmental variability. Prediction Errors for MB and MF Systems The arbitration mechanism is driven by two comple- mentary prediction errors associated with the MB and MF learning systems. The MB system relies on the SPE, which captures errors in state-transition predictions, while the MF system relies on the RPE, reflecting errors in reward prediction. These two signals jointly inform the arbitration process by quantifying the reliability of planning-based versus habitual control. For an agent executing action a i,t from state s i,t and transitioning to s i,t+1 , the SPE is defined as SPE(s i,t , a i,t , s i,t+1 ) = 1[M(s i,t , a i,t ) =∅] + 1[M(s i,t , a i,t )̸=∅] d M ˆ s i,t+1 , s i,t+1 d max ,(35) where d M (·,·) denotes the Manhattan distance between predicted and actual next states [51], d max denotes the maximum achievable Manhattan distance of the environ- ment. 8 Furthermorew, the first indicator ensures SPE = 1 when no model entry exists for (s i,t , a i,t ). A low SPE 8 The Manhattan distance is chosen due to the grid-world structure, where movements occur along axis-aligned edges. Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.11 indicates accurate MB predictions, while a high SPE suggests model inaccuracy or environmental stochasticity. For the MF system, the RPE corresponds to the TD error RPE(s i,t , s i,t+1 ) = δ Q,i,t .(36) Low absolute RPE values indicate that the MF system accurately predicts future rewards, whereas large absolute values reflect substantial prediction errors due to reward changes or insufficient learning. By jointly analyzing SPE and RPE through Bayesian inference, the arbitration mechanism determines the rel- ative reliability of MB and MF systems at each time step, enabling adaptive switching between deliberative planning and habitual, MF control. Dirichlet Reliability Model To compute the reliability scores χ k,i,t for each subsystem k ∈ MB, MF, we maintain a Dirichlet distribution over three prediction-error categories. More specifically, for any prediction error PE (either SPE or RPE), the category is assigned as [49], [50] c(PE) = 0, |PE| < ζ(zero error), 1, PE <−ζ(negative error), 2, PE > ζ(positive error). (37) where ζ denotes a symmetric threshold around zero. This mapping captures whether the system’s predic- tion was essentially correct (category 0) or whether it produced significant underestimates (category 1) or over- estimates (category 2). The threshold ζ controls sensitiv- ity: smaller values make the arbitration mechanism more reactive, while larger values provide robustness against noise and small fluctuations. Letλ k,i,t = [λ k,i,t,0 , λ k,i,t,1 , λ k,i,t,2 ] denote the vec- tor of concentration parameters for system k, where each component λ k,i,t,j counts how many times error category j has been observed. Once a prediction error PE k is observed, we compute its category index j = c(PE k ) and update the correspond- ing Dirichlet parameter: λ k,i,t,j ← λ k,i,t,j + 1.(38) This update constitutes a Bayesian filter that accumu- lates evidence on the frequency of each error type. The normalized vectorθ k,i,t = [θ k,i,t,0 , θ k,i,t,1 , θ k,i,t,2 ] rep- resents the probability distribution over error categories for system k. Under the Dirichlet posterior, the mean and variance of θ k,i,t,0 (probability of zero error) are given by E[θ k,i,t,0 ] = λ k,i,t,0 Λ k,i,t , Var[θ k,i,t,0 ] = λ k,i,t,0 (Λ k,i,t − λ k,i,t,0 ) Λ 2 k,i,t (Λ k,i,t + 1) , (39) where Λ k,i,t = P 2 j=0 λ k,i,t,0 is the sum of counts. A subsystem is considered reliable when it frequently produces negligible errors (high E[θ k,i,t,0 ]) and when such estimates are stable (low variance). Motivated by this 0 4 8 12 16 20 24 y [m] Target Agent 1 Agent 2 Agent 3 Agent 4 04812162024283236 x [m] Fig. 2: Simulation environment for autonomous agent network localization. intuition, the reliability score of subsystem k is defined as χ k,i,t = E 2 [θ k,i,t,0 ] Var[θ k,i,t,0 ] = λ k,i,t,0 (Λ k,i,t + 1) Λ k,i,t − λ k,i,t,0 .(40) A higher value of χ k,i,t indicates greater confidence in system k. In the considered navigation task, this reliability-based arbitration implies that, during early ex- ploration, both MF and MB systems may be unreliable due to sparse experience and inaccurate models, resulting in mixed control. As experience accumulates, improved transition prediction and reduced state prediction errors are expected to increase the reliability of the MB system, whereas convergence of reward prediction errors favors MF control in later stages. These dynamics are validated through the case study presented in the next section. V. Case Study: A Localization Problem In this section, we present performance results to evaluate the performance of the proposed PIT MB/MF learning approach. As discussed earlier, the autonomous agents are tasked with navigating an environment to search for a static target. The mission is deemed success- ful once the PEB associated with the target’s location falls below a predefined threshold (PEB ⋆ = 0.5 m), indicating accurate and reliable target localization. A. Simulation Environment and Parameter Configuration The environment is modeled as a discrete 2D grid of 24 × 36 cells, each measuring (1× 1) m 2 . The layout, illustrated in Fig. 2, includes interior walls and obstacles (shown as black cells), along with designated “gate” zones (highlighted in yellow) and GPS-denied areas (shown in grey). A network of N = 4 autonomous agents is deployed, with initial positions at positions p 1,0 = (10, 20) m, p 2,0 = (2, 16) m, p 3,0 = (4, 6) m, and p 4,0 = (12, 2) m. A single active target, emitting periodic bea- con signals, is located at position p T = (30, 12) m. 12IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. Each agent can select an action from a discrete set A: up, down, left, right, hover. Agent movement is con- strained by the grid boundaries, walls, and other agents, with collision avoidance resolved using (13). The reward is shaped by the received signal strength indicator (RSSI) computed using (11). The path loss exponent is set to η = 2 when the agent-target link is in LOS and to η = 3.5 otherwise. The transmitting power is set to −10 dBm, with both the transmitter and receiver employing antennas with 2 dBi gain. The signal occupies a 1 MHz bandwidth centered at 2.4 GHz, and the receiver noise figure is 10 dB. The wall losses L walls,i,t are set to 25 dB for all agents i and time instants t, and only the room containing the target is considered to be in the LOS condition. If the agent is in a GPS-denied area, its position estimate is affected by increased localization uncertainty, modeled as in (21), where the position error covariance is assumed diagonal with (Σ GPS,i,t ) 1,1 = (Σ GPS,i,t ) 2,2 = 100 m 2 . Each agent is trained with the proposed model for N episodes = 1200 episodes with a maximum step of N steps = 800. The decaying learning rates and discount factor for both the off-policy Q-learning and Pavlovian modulation are identical. Moreover, action selection is governed by a softmax policy with a decaying temperature κ i,t . Other hyperparameters are listed in Tab. I. The learning process is guided by a dual-reward system comprising instrumental and Pavlovian rewards, as described in Sec. E. In simulations, the penalty for collision in (13) is set to λ col = 1.2, the terminal goal reward is set to R goal = 100, and Pavlovian rewards are defined to encode innate, cue-driven tendencies that bias the agent toward advantageous regions and away from structurally risky ones. Gates are assigned a pos- itive Pavlovian reward because they reliably predict a rapid reduction of localization error, whereas GPS-denied regions receive negative value due to their detrimental impact on measurement precision and navigational safety. Specifically, the Pavlovian reward elements are defined as r gate,i,t+1 = 8, ∀i,∀t, r GD,i,t+1 =−8, ∀i,∀t. B. PIT MF Learning: Performance Assessment We first analyze Pavlovian and instrumental MF learn- ing in isolation to assess how cue-driven modulation shapes early exploration and affects agents’ navigation before introducing MB planning components. 1. Evolution of Navigation Policies Across Training The evolution of agent behavior is illustrated by examining their trajectories at different stages of training. Fig 3 shows the trajectories of all four agents trained with PIT MF at three representative training episodes: Episode 1, Episode 100, and Episode 1200. One can observe clearly that at Episode 1, the agents exhibit exploratory and inefficient paths, with some be- coming trapped into loops or entering GPS-denied re- gions. This behavior reflects the high-entropy softmax TABLE I: Simulation Parameters ParameterDescriptionValue α Q ,α Q MB ,α V Learning rate: Initial value0.55 Decay rate0.9985 Final value0.09 γ Q ,γ Q MB ,γ V Discount factor0.98 κ 0 Temperature: Initial value1.2 Decay rate0.996 Final value0.03 KPlanning steps2 βPIT balance1 ξ 1 ,ξ 2 Motivational signal weights0.4, 0.4 P MB Initial P MB 0.5 φMotivational signal balancing weight0.6 policy resulting from a large temperature parameter κ i,t . By Episode 100, the agents’ trajectories become increas- ingly oriented toward the target. They tend to avoid GPS-denied areas and pass through gates in a more reliable manner, characterized by a higher consistency of successful gate crossings across episodes and a reduced number of failed attempts or collisions. This behavior shift reflects the influence of Pavlovian learning, which bi- ases exploration toward contextually favorable transitions. While agents relying solely on instrumental learning are eventually able to pass through the gates, such behavior typically emerges after more prolonged exploration and exhibits greater variability across episodes. In contrast, Pavlovian cue modulation facilitates earlier and more con- sistent gate-crossing behavior. At this stage, agents may already reach the terminal condition, although typically via suboptimal paths. By Episode 1200, the agents display highly efficient behavior. They consistently traverse the gates, maintain LOS with the target, avoid collisions, and approach the target to minimize the PEB, completing the task in only 31 steps. 2. Influence of Motivational Signal In addition to the Pavlovian and instrumental learning components, the influence of motivational signal M i,t is also studied. The results are shown in Fig 3d, 3e, and 3f by holding the same seed as previous simulation. At Episode 1, agents exhibit exploratory and largely random behavior, not perform goal-directed behavior as in Fig 3a. By Episode 100, a pronounced divergence emerges: agents incorporating the motivational signal begin to converge more rapidly toward regions of inter- est, particularly the target area, agents without motiva- tional signal continue to display diffuse and inefficient Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.13 Target 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (a) Target 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (b) Target 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (c) Target 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (d) Target 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (e) Target 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (f) Fig. 3: Agent trajectories with PIT MF at (a) Episode 1, (b) Episode 100, (c) Episode 1200 with (top) and without (bottom) motivational signal, under the same seed. As displayed by the results, without motivational signal, agents are more incline to environmental exploration as per humans [52]. exploration patterns. At Episode 1200, agents of both configurations consistently follow direct, collision-free trajectories toward the goal. These findings are consistent with seminal works in neuroscience like [52], [53], which demonstrated that rats allowed to explore a maze without hunger motivation exhibited diffuse exploratory behavior but later displayed rapid, goal-directed navigation once motivation was intro- duced. In particular, similar to the interpretation of latent learning and cognitive map formation in [53] our results suggest that the motivational signal does not merely enhance reward sensitivity but reorganizes exploratory dynamics into structured, goal-pursuit strategies, thereby accelerating convergence under resource constraints. 3. Instrumental and Pavlovian Values The combined value function ω(s i,t , a i,t ) in (24), com- puted at the end of training, reveals how the two learn- ing systems cooperate to produce the observed optimal behavior, offering insight into the underlying decision- making process. To examine these contributions, we plot separately the instrumental Q-values and the Pavlovian values. Fig 4 shows the maximum instrumental action-value for each agent, max a Q MF (s i,t , a). The resulting Q-table heatmaps for the directional actions display strong posi- tive values in regions where a given action leads directly toward the high-value area surrounding the target. This structure provides the fine-grained action selection re- quired for the direct paths observed in Fig. 3c. These results confirm that the instrumental learning system successfully estimates the expected cumulative reward associated with reaching the goal from each state, thereby forming a reliable solution that guides the agent toward the extrinsic reward. Fig 5 illustrates the Pavlovian state value V 4 (s) (com- puted via (6)) for Agent 4 at several stages of training. Un- like standard RL value functions, which typically encode proximity to the goal, the Pavlovian value evolves into a “risk–reward potential field” shaped by environmental radio cues. The heatmaps exhibit distinct local maxima at gate locations and local minima in GPS-denied regions. These values act as intrinsic shaping signals; for example, the negative potential in GPS-denied regions reduces the likelihood of selecting actions that enter these regions, effectively pruning the search space for the instrumental learner. C. PIT MF-MB Learning Algorithm: Performance Assessment In this subsection, we consider the PIT MF-MB (Hybrid) learning approach, and we compare it with the following baselines: • Instrumental MF: A standard MF RL agent using the Q-learning update in (4) and a softmax policy without Pavlovian modulation. Action selection is driven solely by the instrumental action-value func- tion Q MF . • PIT MF: A MF RL agent in which action selection is influenced by both the instrumental critic and the Pavlovian value function through the modulation term g a (V ) in (24). This corresponds to the PIT MF RL approach described in Sec. I. 14IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. (a)(b)(c)(d) Fig. 4: Maximum Q-values heatmaps max a Q MF (s i,t , a) for PIT MF of each agent at the end of the final training episode. Each column, from left to right, corresponds to a different agent. (a)(b)(c)(d) Fig. 5: Pavlovian value heatmaps of Agent 4 at various stages, namely, Episode 10, Episode 50, Episode 100, Episode 1200, respectively. • Instrumental MF-MB: A Dyna-Q agent that com- bines the instrumental Q MF and MB Q MB estimates as in (32), but with no Pavlovian contribution. • PIT MF-MB (Hybrid): The full proposed algorithm, where the hybrid Q hybrid in (32) integrates MF and MB value estimates, and action selection is further modulated by Pavlovian predictions through g a (V ). These four variants allow us to isolate the individual contributions of (i) Pavlovian modulation, (i) MB plan- ning, and (i) their combined effect within the hybrid approach. 1. Dynamic Arbitration between MB and MF We first examine the effect of integrating MB planning with MF learning through the Bayesian arbitration mech- anism. This analysis highlights how the balance between MB and MF control evolves during training. Fig 6 illustrates the evolution of the arbitration prob- ability P MB , averaged across agents, over the course of training for both the Instrumental MF-MB and the pro- posed PIT MF-MB (Hybrid) architectures. This probabil- ity reflects the relative reliance placed on the MB system and the MF system in action selection, as determined by the Bayesian arbitration mechanism described in Section IV-D. In the early stages of training, both systems exhibit low arbitration probabilities, indicating that neither MB nor MF control is yet dominant. The reason is the internal models are incomplete and value estimates are still noisy. As training progresses, a key divergence emerges between two architectures. In the Instrumental MF-MB baseline, the arbitration probability rises steadily and 03006009001,200 0.7 0.8 0.9 1 Episode ˆ P MB Instrumental MF-MB PIT MF-MB (Hybrid) Instrumental MF-MB Agents 1,2,3,4 PIT MF-MB (Hybrid) Agent 2 PIT MF-MB (Hybrid) Agents 1,3,4 Fig. 6: P MB as a function of episodes. P MB averaged over the number of agents (denoted as ˆ P MB ) between Instrumental MF-MB (blue solid line) and the proposed PIT MF-MB (Hybrid) (red solid line); and P MB between agents with Instrumental MF-MB and the proposed PIT MF-MB (Hybrid) algorithm. The shaded areas bound the minimum and maximum values across the agents. stabilizes at a relatively high value. This indicates that the MB system becomes increasingly favored over the MF system. The underlying reason lies in the persistent unreliability of the MF learner: without Pavlovian modu- lation, the agent frequently explores uncertain or high-risk regions (e.g., GPS-denied areas), leading to high reward prediction errors (RPEs). In contrast, the MB system, once its transition model becomes sufficiently accurate, yields Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.15 low state prediction errors (SPEs), making it the more reliable controller from the perspective of the arbitrator. In contrast, the PIT MF-MB (Hybrid) architecture exhibits a markedly different arbitration profile. Although P MB also increases during the mid-training phase, it eventually stabilizes at a lower value than that observed for the instrumental baseline. This outcome may initially appear counterintuitive, as one might expect Pavlovian bias to further enhance MB reliability. However, it reflects a fundamental property of the proposed hybrid design, that is, Pavlovian conditioning improves the reliability of the MF system itself, thereby reducing the need for MB dominance in later stages. By embedding Pavlovian value signals directly into the action selection policy, the hybrid agent learns to avoid high-risk regions and exploit perceptually informative cues (e.g., gates) early in training. This leads to more con- sistent and predictable transitions, which in turn stabilizes MF value estimates and reduces RPEs. As a result, the MF system becomes a more trustworthy controller, and the arbitrator no longer needs to rely predominantly on the MB system. The MB planner remains available and accurate, but the arbitration mechanism naturally shifts toward a more balanced configuration when both systems are reliable. This interpretation is further supported by the agent- specific arbitration profiles. Agents operating in more structured or constrained regions (e.g., Agent 2 in Fig 6) continue to exhibit lower MB reliance for longer periods, reflecting its higher reliance for Pavlovian bias guidance. Meanwhile, agents in more open areas (e.g., Agent 1) transition more quickly to MB-dominated control. These observations demonstrate that arbitration is a context- sensitive, agent-specific learning process. 2. Impact of Pavlovian Modulation to Agents’ Navigation Fig. 7 compares the final trajectories of the proposed hybrid framework against a standard Instrumental MF- MB baseline. The baseline agents successfully locate the target but generate paths that graze the GPS-denied re- gion, incurring higher measurement variance. This occurs because the standard Q-learning update slowly propagates the negative reward from the GPS-denied region. In contrast, the proposed PIT MF-MB approach (Fig. 7b) executes a decisive avoidance maneuver well before reaching the hazardous zone. This is a direct manifes- tation of the PIT phenomenon. GPS-denied regions act as CS predicting punishment, leading to higher PEB values. Through the action-dependent modulation, any action leading toward these states is penalized in the softmax logits, creating a repulsive gradient. This cue- driven bias operates immediately upon perceiving the predictive region, enabling rapid, reflex-like avoidance without requiring expensive trial-and-error learning of the long-term consequences. 3. Comparative Evaluation Finally, having characterized the individual roles of Pavlovian modulation and MB arbitration, we compare 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (a) 04812162024283236 x [m] 0 4 8 12 16 20 24 y [m] Target (b) Fig. 7: Agent final trajectories with (a) Instrumental MF- MB, (b) PIT MF-MB (Hybrid) learning approaches. 0200400600800 0 200 400 600 800 Episode Steps Instrumental MF w/o M i,t Instrumental MF w/ M i,t PIT MF w/o M i,t PIT MF w/ M i,t Instrumental MF-MB w/o M i,t Instrumental MF-MB w/ M i,t PIT MF-MB (Hybrid) w/o M i,t PIT MF-MB (Hybrid) w/ M i,t Fig. 8: Learning rates expressed as steps per episodes and for various combinations of learning types. “w/” and “w/o” refer to the presence, or not, of the motivational signal. the performance of all algorithmic variants. This compar- ison isolates the contribution of each subsystem, Pavlo- vian, MF, and MB, and quantifies their combined effect within the proposed hybrid architecture. 16IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No. Fig. 8 shows the number of steps needed to achieve the PEB goal as a function of the number of episodes, averaged over 40 Monte Carlo simulations. Results are shown for four algorithmic configurations, each presented both with and without Pavlovian modulation. We can observe that the proposed PIT MF-MB (hybrid) approach (e.g., the black solid line) reaches a competent policy significantly earlier than Instrumental MF (e.g., the red solid line). This confirms the hypothesis that a cue-driven learning process accelerates exploration toward low-risk, information-rich regions. We can also observe that MB approaches produce a more deterministic final policy faster than Instrumental MF and PIT MF approaches. The hybrid PIT MF-MB algorithm achieves the fastest stabilization of around 200 episodes. The results above demonstrate that integrating Pavlo- vian, instrumental, and MB components yields capabili- ties that none of the individual subsystems can achieve on its own. The Pavlovian modulation provides immediate, cue-driven shaping that accelerates exploration toward perceptually informative structures, while simultaneously discouraging transitions into geometrically hazardous ar- eas such as GPS-denied regions. This shaping substan- tially reduces the burden on the instrumental learner and leads to faster discovery of high-value areas, such as LOS regions. The instrumental component contributes fine-grained action discrimination and long-term reward estimation, supplying the precise directional gradients required for efficient multi-agent localization. The MB planner further enhances stability by propagating con- sistent predictions, smoothing local inconsistencies in instrumental values, and reducing dependence on noisy or incomplete real-world experience. The arbitration mecha- nism coordinates these elements dynamically, relying on the planner only when its internal model is reliable, and allowing the MF system to dominate when rapid, stable exploitation is sufficient. Together, these findings indi- cate that the cognitive-inspired hybrid PIT MB and MF architecture provides a principled approach to enhance autonomous decision-making in complex environments. Moreover, across all architectures, the inclusion of the motivational signal consistently accelerates convergence, as evidenced by the lower number of steps required to reach the PEB goal. This effect is particularly pronounced in the Instrumental MF and PIT MF configurations The performance gap underscores the role of the motivational signal as an internal drive mechanism that dynamically adjusts behavior based on resource depletion and mission urgency. By introducing a state-dependent cost term, the motivational signal penalizes inefficient actions and biases exploration toward goal-relevant regions, thereby reducing time spent in low-utility areas. Notably, even in the PIT MF-MB (Hybrid) architecture, which already benefits from Pavlovian biasing and MB planning, and the addition of the motivational signal yields further improvement, suggesting that internal drive modulation provides a complementary and non-redundant contribu- tion to learning. These results corroborate the biological inspiration underlying the proposed framework: just as hunger or fatigue shapes decision-making in humans, the motivational signal enables autonomous agents to adapt their exploration-exploitation trade-off under in- ternal constraints, leading to faster and more resource- efficient navigation. VI. Conclusions In this work, we introduced a human-inspired hy- brid RL framework that unifies Pavlovian conditioning, instrumental MF learning, and MB planning to enhance autonomous agents’ navigation in uncertain environments. We introduced the concept of contextual radio cues, defined as specific georeferenced states of the environ- ment acting as conditioned stimuli that predict task- relevant outcomes (i.e., increases or decreases in the PEB) based on their inherent electromagnetic characteristics. Then, leveraging the principle of PIT, the framework uses these contextual radio cues to shape action selection by biasing the instrumental-only policy using an action- dependent value function. This PIT-based policy enables agents to exploit contextual information to approach task- informative regions and proactively avoid hazardous ar- eas. A Bayesian arbitration mechanism further stabilizes behavior by dynamically balancing habitual control (MF learning) and deliberative planning (MB control) ac- cording to reliability scores. Each agent performs this arbitration locally based on its own experience and inter- nal signals. Although agents learn independently and do not explicitly share learned value functions or policies, they interact through the shared environment during the localization task. Simulation results for a multi-agent localization task show that this integrated design acceler- ates convergence and improves safety compared to each subsystem operating in isolation. Future works will extend the architecture to high- dimensional continuous state-action spaces through deep RL, strengthen multi-agent cooperation through commu- nication of Pavlovian values and internal model updates, and validate the approach on physical platforms to evalu- ate robustness under real-world sensing and environmen- tal variability. REFERENCES [1] A. Guerra et al., “Reinforcement Learning for Joint Detection and Mapping Using Dynamic UAV Networks,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 3, p. 2586–2601, 2023. [2] I. Guvenc et al., “Detection, Tracking, and Interdiction for Am- ateur Drones,” IEEE Commun. Mag., vol. 56, no. 4, p. 75–81, 2018. [3] A. Guerra et al., “Networks of UAVs of Low Complexity for Time- Critical Localization,” IEEE Aerosp. Electron. Sys. Mag., vol. 37, no. 10, p. 22–38, 2022. [4] R. Azzam et al., “Learning-based navigation and collision avoid- ance through reinforcement for UAVs,” IEEE Trans. Aerosp. Electron. Syst., vol. 60, no. 3, p. 2614–2628, 2024. [5] M. Dorigo, G. Theraulaz, and V. Trianni, “Swarm robotics: Past, Present, and Future [Point of View],” Proc. IEEE, vol. 109, no. 7, p. 1152–1165, 2021. Submitted to IEEE Transactions on Aerospace and Electronic Systems: J. Shan et al.17 [6] A. Mondal et al., “Multiagent reinforcement learning for offload- ing cellular communications with cooperating UAVs,” IEEE Trans. Aerosp. Electron. Syst., vol. 61, no. 4, p. 9344–9358, 2025. [7] S. A. Soleymani et al., “Multiagent q-learning with particle fil- tering for UAV tracking in open-RAN environment,” IEEE Trans. Aerosp. Electron. Syst., vol. 61, no. 4, p. 10 439–10 458, 2025. [8] A. M. Ahmed et al., “A reinforcement learning based approach for multitarget detection in massive mimo radar,” IEEE Trans. Aerosp. Electron. Syst., vol. 57, no. 5, p. 2622–2636, 2021. [9] D. Lee et al., “Optimization for reinforcement learning: From a single agent to cooperative agents,” IEEE Signal Process. Mag., vol. 37, no. 3, p. 123–135, 2020. [10] M.-A. Lahmeri, M. A. Kishk, and M.-S. Alouini, “Artificial intelligence for UAV-enabled wireless networks: A survey,” IEEE Open J. Commun. Soc., vol. 2, p. 1015–1040, 2021. [11] G. Fontanesi et al., “A deep-N beamforming approach for dual function radar-communication THz UAV,” IEEE Trans. Veh. Technol., vol. 74, no. 1, p. 746–760, 2025. [12] S. Haykin et al., “Cognitive control,” Proc. IEEE, vol. 100, no. 12, p. 3156–3169, 2012. [13] P. Dayan and B. W. Balleine, “Reward, motivation, and reinforce- ment learning,” Neuron, vol. 36, no. 2, p. 285–298, 2002. [14] D. Hassabis et al., “Neuroscience-inspired artificial intelligence,” Neuron, vol. 95, no. 2, p. 245–258, 2017. [15] N. M. Afshar et al., “Reward-mediated, model-free reinforcement- learning mechanisms in pavlovian and instrumental tasks are related,” J. Neurosci., vol. 43, no. 3, p. 458–471, 2023. [16] S. Ghirlanda, J. Lind, and M. Enquist, “A-learning: A new formulation of associative learning theory,” Psychon. Bull. Rev., vol. 27, no. 6, p. 1166–1194, 2020. [17] T. Tsurumi et al., “Online reinforcement learning of state repre- sentation in recurrent network supported by the power of random feedback and biological constraints,” Elife, vol. 14, 2025. [18] P. Dayan and K. C. Berridge, “Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation,” Cogn. Affect. Behav. Neurosci., vol. 14, no. 2, p. 473–492, 2014. [19] A. R. Delamater and S. Oakeshott, “Learning about multiple attributes of reward in Pavlovian conditioning,” Ann. N. Y. Acad. Sci., vol. 1104, no. 1, p. 1–20, 2007. [20] M. J. Robinson and K. C. Berridge, “Instant transformation of learned repulsion into motivational “wanting”,” Curr. Biol., vol. 23, no. 4, p. 282–289, 2013. [21] E. R. Pool et al., “Behavioural evidence for parallel outcome- sensitive and outcome-insensitive Pavlovian learning systems in humans,” Nat. Hum. Behav., vol. 3, no. 3, p. 284–296, 2019. [22] S. W. Lee and B. Seymour, “Decision-making in brains and robots—the case for an interdisciplinary approach,” Curr. Opin. Behav. Sci., vol. 26, p. 137–145, 2019. [23] P. Mahajan et al., “Balancing safety and efficiency in human decision making,” bioRxiv, p. 2024–01, 2024. [24] S. Elfwing and B. Seymour, “Parallel reward and punishment control in humans and robots: Safe reinforcement learning using the MaxPain algorithm,” in Proc. Joint IEEE Int. Conf. Develop. Learn. Epigenet. Robot. (ICDL-EpiRob), 2017, p. 140–147. [25] J. Wang, S. Elfwing, and E. Uchibe, “Modular deep reinforcement learning from reward and punishment for robot navigation,” Neural Netw., vol. 135, p. 115–126, 2021. [26] E. Cartoni, S. Puglisi-Allegra, and G. Baldassarre, “The three principles of action: a Pavlovian-instrumental transfer hypothesis,” Front. Behav. Neurosci., vol. 7, p. 153, 2013. [27] E. Cartoni et al., “A Bayesian model for a Pavlovian-instrumental transfer hypothesis,” in Proc. First Multidisciplinary Conf. Rein- forcement Learning and Decision Making, Oct. 2013. [28] L. Baird, “Reinforcement learning in continuous time: advantage updating,” in Proc. 1994 IEEE Int. Conf. Neural Netw. (ICNN), vol. 4, 1994, p. 2448–2453. [29] P. Dayan and C. Watkins, “Q-learning,” Mach. Learn., vol. 8, no. 3, p. 279–292, 1992. [30] H. Jung et al., “Multi-Agent AMIX-DAPG of Dual-Arm Robot for Long Horizon Lifecare Tasks,” in Proc. 2025 Int. Conf. Artif. Intell. Inf. Commun. (ICAIIC), 2025, p. 0866–0870. [31] Z. Li et al., “Continuous advantage learning for minimum-time trajectory planning of autonomous vehicles,” Sci. China Inf. Sci., vol. 67, no. 7, p. 172206, 2024. [32] F. Guidi et al., “A Cognitive Framework for Autonomous Agents:TowardHuman-InspiredDesign,”arXivpreprint arXiv:2601.16648, 2026. [33] R. S. Sutton, A. G. Barto et al., Reinforcement learning: An introduction. MIT press Cambridge, 1998, vol. 1, no. 1. [34] K. Juechems and C. Summerfield, “Where does value come from?” Trends Cogn. Sci., vol. 23, no. 10, p. 836–850, 2019. [35] A. D. Craig, “Interoception: The sense of the physiological con- dition of the body,” Curr. Opin. Neurobiol., vol. 13, p. 500–505, 2003. [36] M. Saeidi et al., “Pavlovian-Inspired Cue-Outcome Association for Autonomous Agent Navigation,” in Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Process. Workshops (ICASSPW), 2026, p. 1–5. [37] E. D. Boorman, M. F. Rushworth, and T. E. Behrens, “Ven- tromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multi-alternative choice,” J. Neurosci., vol. 33, p. 2242–2253, 2 2013. [38] K. Torii et al., “Hypothalamic control of amino acid appetite,” Ann. N. Y. Acad. Sci., vol. 855, p. 417–425, 1998. [39] T. Ono et al., “Hypothalamic neuron involvement in integration of reward, aversion, and cue signals,” J. Neurophysiol., vol. 56, p. 63–79, 1986. [40] J. Panksepp and J. Moskal, Dopamine and SEEKING: Subcortical ”reward” systems and appetitive urges. Psychology Press., 2008, p. 67–87. [41] J. Bosulu et al., “wanting” versus “needing” related value: An fmri meta-analysis,” Brain Behav., vol. 12, 9 2022. [42] P. Cisek, “Resynthesizing behavior through phylogenetic refine- ment,” Atten. Percept. Psychophys., vol. 81, p. 2265–2287, 10 2019. [43] M. A. Salichs and M. Malfaz, “A new approach to modeling emotions and their use on a decision-making system for artificial agents,” IEEE Trans. Affect. Comput., vol. 3, no. 1, p. 56–68, 2012. [44] D. Dardari et al., “Ranging With Ultrawide Bandwidth Signals in Multipath Environments,” Proceedings of the IEEE, vol. 97, no. 2, p. 404–426, 2009. [45] M. Guitart-Masip et al., “Go and no-go learning in reward and punishment: interactions between affect and effect,” Neuroimage, vol. 62, no. 1, p. 154–166, 2012. [46] J. C. Swart et al., “Catecholaminergic challenge uncovers distinct Pavlovian and instrumental mechanisms of motivated (in) action,” Elife, vol. 6, p. e22169, 2017. [47] J. F. Cavanagh et al., “Frontal theta overrides pavlovian learning biases,” J. Neurosci., vol. 33, no. 19, p. 8541–8548, 2013. [48] H. M. Dorfman and S. J. Gershman, “Controllability governs the balance between pavlovian and instrumental action selection,” Nat. Commun., vol. 10, no. 1, p. 5826, 2019. [49] N. D. Daw, Y. Niv, and P. Dayan, “Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control,” Nat. Neurosci., vol. 8, no. 12, p. 1704–1711, 2005. [50] S. W. Lee, S. Shimojo, and J. P. O’doherty, “Neural computa- tions underlying arbitration between model-based and model-free learning,” Neuron, vol. 81, no. 3, p. 687–699, 2014. [51] S. Russell, P. Norvig, and A. Intelligence, “A modern approach,” Artificial Intelligence. Prentice-Hall, Egnlewood Cliffs, vol. 25, no. 27, p. 79–80, 1995. [52] E. C. Tolman and C. H. Honzik, “Introduction and removal of reward, and maze performance in rats.” University of California publications in psychology, 1930. [53] E. C. Tolman, “Cognitive maps in rats and men.” Psychol. Rev., vol. 55, no. 4, p. 189, 1948. 18IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMSVOL. , No.