Paper deep dive
tinyDSM: A Framework for Skill Modeling and Development for Resource-Constrained Millirobots
Markus D. Kobelrausch, Michael Miedler, Axel Jantsch
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/19/2026, 5:09:38 AM
Summary
The paper introduces tinyDSM, a framework for developmental skill modeling and learning in resource-constrained millirobots. It combines intrinsic motivation with reinforcement learning to enable autonomous skill acquisition, utilizing a hierarchical knowledge graph and kinematic reasoners. The system is demonstrated on a Raspberry Pi Pico-based millirobot, showing progression from simple motor skills to complex geometric patterns within 15 minutes.
Entities (7)
Relation Signals (6)
tinyDSM → employs → Reinforcement Learning
confidence 95% · Reinforcement learning algorithms guide the agent's skill acquisition and adaptation through the interplay of our proposed tinyDSM...
tinyDSM → runson → millirobot
confidence 95% · In our experiments, we use a resource-constrained millirobot... that integrates all described features and capabilities...
tinyDSM → uses → Intrinsic Motivation
confidence 95% · Reinforcement learning algorithms guide the agent's skill acquisition and adaptation through the interplay of our proposed tinyDSM, which integrates intrinsic motivation and fitness-based assessment.
tinyDSM → uses → Knowledge Graph
confidence 95% · It uses a hierarchical knowledge graph and kinematic reasoners to model and evaluate simple and advanced motion related skills.
millirobot → uses → Raspberry Pi Pico
confidence 95% · In our experiments, we use a resource-constrained millirobot with a volume of 36 cm^3 with a Raspberry Pi Pico 32-bit microcontroller (RP2040)...
Knowledge Graph → models → Sensor-Actuator Space
confidence 90% · The foundation of our cognitive system model is a Knowledge Graph (KG) with semantic features... that contextualizes the system components with dependencies... Currently, the robot can only draw conclusions about its hierarchical relationships with skills and the Sensor and Actuator Space (SAS).
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In this study, we investigate developmental mechanisms that enable small, resource-constrained systems such as cm-sized millirobots to autonomously explore, learn, and adapt their capabilities throughout their lifespan. Reinforcement learning algorithms guide the agent's skill acquisition and adaptation through the interplay of our proposed tinyDSM, which integrates intrinsic motivation and fitness-based assessment. We strive for minimal, hard-wired skills while encouraging the open-ended development of new skills. A key emphasis in our approach is to encode minimal a-priori general knowledge, which serves as a foundational starting point for the system as it further learns system-specific dependencies from the initial knowledge provided. Thus, by design, our approach attempts to cover very generic application domains. The methodology is based on (a) developmental mechanism with intrinsic motivation, and (b) a cognitive architecture (knowledge, reasoning, learning), while (c) utilizing minimal resources. It uses a hierarchical knowledge graph and kinematic reasoners to model and evaluate simple and advanced motion related skills. In our experiments, we use a resource-constrained millirobot with a volume of 36 cm^3 with a Raspberry Pi Pico 32-bit microcontroller (RP2040) that integrates all described features and capabilities except the camera system in 9 kB. Starting with learning the most elementary motor skills the millirobot autonomously progresses from simple linear and angular movements to complex geometric patterns within 15 minutes. To complement the physical experiments, we perform a simulation-based analysis that enables systematic comparisons across learning algorithms and intrinsic motivation parameters.
Tags
Links
- Source: https://arxiv.org/abs/2608.17596v1
- Canonical: https://arxiv.org/abs/2608.17596v1
Trouble viewing inline? Open PDF directly →
Full Text
85,798 characters extracted from source content.
Expand or collapse full text
tinyDSM: A Framework for Skill Modeling and Development for Resource-Constrained Millirobots Markus D. Kobelrausch Michael Miedler Axel Jantsch Thanks: Markus D. Kobelrausch, Michael Miedler, and Axel Jantsch are with the Institute of Computer Technology, TU Wien, Vienna, Austria. Corresponding author: Markus D. Kobelrausch (e-mail: markus.kobelrausch@tuwien.ac.at). Abstract In this study, we investigate developmental mechanisms that enable small, resource-constrained systems such as cm-sized millirobots to autonomously explore, learn, and adapt their capabilities throughout their lifespan. Reinforcement learning algorithms guide the agent’s skill acquisition and adaptation through the interplay of our proposed tiny Developmental Skill Method (tinyDSM), which integrate intrinsic motivation and fitness-based assessment. We strive for minimal, hard-wired skills while encouraging the open-ended development of new skills. A key emphasis in our approach is to encode minimal a-priori general knowledge, which serves as a foundational starting point for the system as it further learns system-specific dependencies from the initial knowledge provided. Thus, by design, our approach attempts to cover very generic application domains. The methodology is based on (a) developmental mechanism with intrinsic motivation, and (b) a cognitive architecture (knowledge, reasoning, learning), while (c) utilizing minimal resources. It uses a hierarchical knowledge graph and kinematic reasoners to model and evaluate simple and advanced motion related skills. In our experiments, we use a resource-constrained millirobot with a volume of 36 cm336\,cm^3 with a Raspberry Pi Pico 32 bit microcontroller (RP2040) that integrates all described features and capabilities except the camera system in 9 kB9\,kB. Starting with learning the most elementary motor skills the millirobot autonomously progresses from simple linear and angular movements to complex geometric patterns within 15 minutes. To complement the physical experiments, we perform a simulation-based analysis that enables systematic comparisons across learning algorithms and intrinsic motivation parameters. Index Terms: tiny robot learning, developmental robotics, cognitive architectures, reinforcement learning, low-energy mobile robots I Introduction DEVELOPMENTAL robotics investigates mechanisms that enable a robot to continuously explore its environment, learn from its experiences and adapt to changes throughout its lifetime [22]. Cognitive approaches aim to use a synthetic approach that constructs cognitive functions in a developmentally appropriate manner that is inspired by developmental principles and mechanisms observed in children and animals [6]. Physical embodiment enables the structuring of information through interactions with the environment, whereby the hypothesized developmental model can vary in complexity, ranging from body representation, perception, motor skills and social behaviour to linguistic interaction. Such cognitive approaches are studied in the robotics literature at different levels of complexity, with the learning of such models aiming, for example, to acquire skills for navigating a given environment [2]. Robot learning has received a significant boost from Machine Learning (ML), with a trend towards advanced robots with methods that process an enormous amount of information and consequently require a lot of resources in terms of memory, time and energy [39]. As a result, they require significant amounts of prior knowledge and energy to operate effectively. Tiny robot learning deals with the deployment of ML on resource-constrained low-cost autonomous robots. The roots lie in the intersection of embedded systems, robotics and ML with the subject to challenges from size, weight, area, and power constraints along with sensor, actuator and compute hardware limitations [25]. These lightweight robots (weighing under 500 g500\,g [7]) can operate in small spaces and offer promising solutions for a wide range of applications, from emergency search and rescue [11] to routine monitoring and maintenance of infrastructure and equipment [10]. They have limited sensors and actuators and have to learn computationally demanding, complex and robust behaviours in different application spaces. The deployment varies across different robot models, system components, tasks and environments. Consequentially, a critical trade-off must be made between energy and memory resources for machine learning and the other system components, such as sensors/actuators and complex behaviour. Our work aims to integrate interdisciplinary fields by addressing the challenge of combining development robotics concepts within the resource constraints of tiny robot learning. It aims to equip millirobots with a cognitive architecture (limited to knowledge, reasoning and learning) that learns competencies in a development-oriented manner and strives for open-ended learning of new skills and knowledge. To better comprehend our scope and goals, let us consider the case of a mobile millirobot that aims to acquire motor skills. In this work, we refer to motor skills more generally as motion skills, emphasizing the mobility of the millirobot. To be very general we aim at a minimal set of general knowledge that serves as a foundational starting point for the robot with: APhysical and Geometric Laws Knowledge about geometric space, and certain physical relations about friction and motion, as manifest in the kinematic reasoner. BSensors and Actuators Sensors provide information about the environment, and actuators influence the environment. The exact impact of actuators is unknown, although it is assumed that actuator effects are sensed by sensors (kinematic reasoning). CSkills, Complex Behaviour, Learner The concept of skill, as defined. An instinctive desire to explore skills, and drive them to high fitness levels, as well as a suitable learning mechanism. DContext Awareness and Knowledge Structure A foundational understanding grounded in a knowledge graph that integrates dependencies for sensors, actuators, and skills. Fig. 1: Assumptions and a-priori knowledge that the millirobot has built-in. Let’s assume the millirobot can move using two motors and senses its motion via an acceleration sensor. It knows which kind of sensors and actuator it has (physical model), but it does not know the effects of actuator commands or anything about the environment - it only has assumptions about sensory information (kinematic reasoner). Initially, the robot queries its knowledge graph based on its available sensor and actuator set and infers competencies that it can potentially develop. We argue knowledge representation and reasoning are essential as it is universally applicable and allow for generic application or reuse in other contexts [27]. As the robot interacts with its environment, it explores its sensorimotor effects and reasons about its motions utilizing its kinematic reasoner, thus it develops an understanding of the relationship between knowledge, perception and action. Through that exploration, it develops basic motion competencies, which gradually evolve into more complex skills. It achieves this by engaging in complex behaviour where it is intrinsically motivated to pursue the most fascinating skills. In psychology, Intrinsic Motivation (IM) is considered a driving cause of autonomous entities to acquire and develop skills, and it is key to their development, as it enables them to effectively deal with problems that arise [35]. The robot continuously assesses its learning progress by evaluating its skills effectiveness (fitness reasoner). This fosters an increasing understanding and expertise, which affects motivation and drives the pursuit of specific skills. Thus, the robot exhibits behaviour that strives for continuous development and improvement, while it adapts to any changes in its environment. In the case of an expansion of its knowledge graph, it tackles the challenge of a novel development while maintaining a balance with improving existing skills. In our experiments, the millirobot develops skills through a hierarchical modelling approach. It begins by discovering movement patterns, stored in the knowledge base, such as angular and linear movements. Subsequently, it learns specific distances and angles based on these patterns. The complex behaviour derived from IM is the focus of this work. We examine various patterns in detail and demonstrate their potential for enabling the development of an agent that operates without a predetermined goal. We analyze the effects of its intrinsically motivated behaviour on learning and investigate how it responds to different situations. Additionally, we assess the accuracy of its movements by studying a compound movement pattern, namely a square. To complement these physical experiments, we also perform a simulation-based analysis to systematically evaluate different learning algorithms and intrinsic motivation configurations. To this end, we propose tinyDSM a framework for millirobots which features • a developmental mechanism with intrinsic motivation, • a cognitive architecture (knowledge, reasoning, learning), • all while utilizing minimal resources. We consider this a worthwhile approach. To minimize prior knowledge and assumptions will facilitate very flexible systems. It will allow the use of accurate or inaccurate sensors and actuators, and to adapt to aging and wear-out effects. Our long-term goal is to provide the robot with general methods that allow it to work with any kind of sensors and actuators in any kind of physical environment. Imagine a wheel-equipped or flying robot, on level plains, rocky or grassy surfaces or even in wet environments and learning any kind of competence, provided it is possible at all (e.g. if the robot has only LEDs but no motors, it cannot learn to move). I Background Our tinyDSM is based on several domains of research, we first review the different areas. The broader domain is developmental robotics, which studies mechanisms that allow robots to learn and adapt independently over time, inspired by developmental principles observed in children and animals [22, 2, 6]. Cognitive architectures such as LRMB[47], ATC-R [34] or SORA [17] model layered cognition, enabling robots to process perceptual data, reason logically, and organize information into conceptual categories [19]. Our approach is similar to SORA as we work within a decision cycle that includes perception, action, memory and reasoning. We differ from SORA in that we do not rely on comprehensive symbolic reasoning that involves working or long-term memory, but instead use a lightweight knowledge structure and explicit motion-related reasoning. This limitation is intentional, as we want to maintain a minimal cognitive architecture that relies on a minimal set of assumptions. Robots benefit from a structured semantic model to understand the real world. Especially ontologies organize knowledge in a way that allows cognitive robots to reason and create understanding [27]. Knowledge bases are popular in service robots modeling complex domain specific knowledge [42]. Frameworks such as KnowRob[46], RoboBrain[37] or BWIBots[15] show success in mastering a variety of complex tasks, ranging all the way to cognitive language skills. KnowRob and RoboBrain focus on knowledge representation with extensive knowledge databases and reasoners for various service-oriented robots. In contrast, our work focuses on a reduced knowledge graph specifically for mobile millirobots. While BWIBots also specializes in navigation, it focuses on a collaborative robot and, unlike us, assumes static task planning. Our approach differs in that we extend to dynamic planning with intrinsic motivation and consider tightly limited resources. Intrinsic motivation (IM) is a central concept in lifelong learning and is understood as the drive to engage in activities for their own sake, primarily for enjoyment and satisfaction, rather than because of external rewards or explicit tasks. In robotics, IM draws inspiration from cognitive science and psychology, where curiosity and self-directed exploration are emphasised in learning [28, 29, 30, 35]. IM is often combined with Hierarchical Reinforcement Learning (HRL) to create a multi-level policy structure [4, 38, 32, 1]. Our approach is also based on hierarchical competence models but differs in the use of an explicit, structured knowledge graph that the agent actively queries to drive its development. Specifically, we derive a difficulty factor from the graph that is incorporated into the intrinsic motivation. We consider novelty, progress, and difficulty in intrinsic motivation, and our specialized computation demonstrates robust and effective behaviors. Moreover, we aim to embed learning mechanisms in the highly constrained resource space of tiny robots [25]. TinyML offers new possibilities with techniques such as quantization, pruning, and clustering to reduce the computational load and memory requirements of machine learning models, enabling them to be deployed on low-resource devices [14, 18, 21, 36]. Although this approach is promising, most TinyML systems rely on offline training and therefore do not fully meet our requirements. In terms of algorithms, Reinforcement Learning (RL) is well suited due to the design of our reinforcement flow in tinyDSM. Algorithms such as Deep Q-Networks (DQN) [23] and Deep Deterministic Policy Gradient (DDPG) [20] are viable candidates. However, these models are large and require significant resources. Among traditional RL algorithms, Q-learning-based algorithms [48] are effective for low dimensional search spaces due to their simplicity and low resource requirements. TinyRL is still in the early stages, but research on deep RL in resource-constrained environments is growing [45, 44]. Such approaches are promising and offer potential for integration into our framework. Besides that, genetic algorithms show also potential for policy optimization [40, 41]. Specific work similar to ours includes the research from Baranes and Oudeyer [3] who employ an intrinsically motivated approach to learning inverse models by actively selecting goals based on learning progress. Forestier et al. [12] extended this by curriculum-based learning which allows the complexity of the goals to be self-organized. In contrast, we extend this approach by also considering novelty and difficulty factors in the intrinsic motivation. Moreover, the development frameworks of Nguyen & Oudeyers [26] and Colas et al. [9] also focus on the idea of skill-based intrinsic motivation for development. In summary, our work integrates concepts from developmental robotics, cognitive architectures, intrinsic motivation, and resource-constrained learning to enable on-device skill learning millirobots. Our work demonstrates that cognitive architectures can effectively scale to function in highly constrained environments, as confirmed by the minimal resource footprints in our experimental results. Due to the strong growth and interest in IoT devices, we see great potential for our approach to reduce customization efforts in this domain. I Developmental Skill Method I-A Definition of SAS, Fitness and Skills I-A1 SAS The Sensor-Actuator Space SAS=(,) SAS=( , ) is a 2-tuple and consists of a set of sensors and a set of actuators . A sensor reading si∈×T→ℝP(si)s_i∈ × T ^P(s_i) is a mapping of a sensor identifier at a given time to a vector of values read from the sensor interface. An actuator command cj∈×T×ℝP(cj)c_j∈ × T×R^P(c_j) is defined by the actuator identifier cjc_j and a set of P(cj)P(c_j), that are passed to the actuator interface at a given time. For instance, for a robot with two motors =cm1,cm2 =\c_m_1,c_m_2\, the two corresponding motor commands each take three parameters, the time of activation t, the rotational force f and the duration τ how long the force is applied: cm1(t,f,τ)c_m_1(t,f,τ) and cm2(t,f,τ)c_m_2(t,f,τ). The robot has one sensor s∈s∈ that provides the translation information s(t)→[x,y,ψ]s(t)→[x,y,ψ] at time t, with x and y denoting the translations in the x- and y-dimensions, respectively, and ψ denoting the yaw angle. I-A2 Fitness An integral element of a skill is the assessment of its quality. It is based on a skill dependent set of quantities. These quantities are often sensor readings, like distance or velocity, but may include outputs of other skills. For a fitness that depends on n quantities with n∈ℕ,n>0,n ,n>0, we define the following vectors. t→=(t0t1…tn−1)o→=(o0o1…on−1)r→=(r0r1…rn−1)e→=(e0e1…en−1) t= pmatrixt_0\\ t_1\\ …\\ t_n-1 pmatrix\\ \> o= pmatrixo_0\\ o_1\\ …\\ o_n-1 pmatrix\\ \> r= pmatrixr_0\\ r_1\\ …\\ r_n-1 pmatrix\\ \> e= pmatrixe_0\\ e_1\\ …\\ e_n-1 pmatrix The target vector t→ t signifies the desired quantities we seek to reach. The range vector r→ r defines the maximum permissible deviation from the target vector. The observation vector o→ o captures the values observed after the execution of an activity, and the error vector e→ e is calculated as follows: ei=|oi−ti|ri,for i=0,1,…,n−1e_i= |o_i-t_i|r_i, i=0,1,…,n-1 The scalar fitness value f∈[0,1]f∈[0,1] is then computed as: f=1−‖e→‖nf=1- || e|| n Putting this together, the fitness F of a skill is a 5-tuple. F=(t→,r→,o→,f(.),fthreshold)F=( t, r, o,f(.),f_threshold) (1) A skill is deemed “learned” if f(.)≥fthresholdf(.)≥ f_threshold. The required threshold can vary but is typically around 0.950.95. Hence, the fitness function, often called the fitness of a skill, evaluates to a single scalar number that represents the quality of an activity. It is used by learning routines to direct the learning process and establishes awareness of the level of competence an agent has at any given time. Fig. 2: Overview of the definition of a skill including its specification, routine and state (during execution). I-A3 Skill As shown in Fig. 2, a skill o consists of three components: a specification, a learning routine and a state. The skill specification defines the skill in terms of a relation between the action space and the sensor space. Formally, it is a 2-tuple DS=(T,F)D_S=(T,F) consisting of a desired transformation of the sensor readings T and the fitness F, as defined above. T(S0,t0,S′,t′)T(S_0,t_0,S ,t ) denotes the desired transformation on the sensor readings, where S0=[s1,s2,…]S_0=[s_1,s_2,...] is the set of sensor readings at time t0t_0, before the first action a1a_1 is applied, and S′=[s1′,s2′,…]S =[s _1,s _2,...] is the set of sensor readings at time t′t after the last action of A has been applied. The learning routine is a learning algorithm that modifies the action sequence with the objective to maximize the fitness. A reinforcement learning algorithm is a typical and good example for a learning routine. The State of a Skill o, X(t)=(A,f¯)X(t)=(A, f) is a 2-tuple consisting of an action sequence A, the current fitness value f¯ f, as evaluated by the fitness function f(.)f(.). A=⟨a1,a2,…⟩A= a_1,a_2,... is the set of commands a∈×ℝP(a)a∈ ×R^P(a) applied in sequence. Whenever the skill should be applied, A is executed. Initially A may be empty and the learning routine has the task to modify A such that the fitness is maximized. We focus on modelling actions that lead to physical actions of the system, but not necessarily motion. For example, consider a robot that has an LED actuator paired with a brightness sensor nearby. When the LED is turned on, it may cause a noticeable change in brightness in the surrounding environment, which the sensor then detects. The objective of the learning algorithm may be to regulate the brightness to a specific target value, enabling the agent to adjust this brightness level based on the correlations it has learned. This scenario exemplifies a lighting skill, with a help task defining the sequence of brightness actions that signal the SOS distress signal, enabling the millirobot to ask for help. Note, that this concept is general and can extend to all kind of physical actions. I-B Knowledge Graph The foundation of our cognitive system model is a Knowledge Graph (KG) with semantic features [8], that contextualizes the system components with dependencies. It reflects the scope of development at a certain point in time. Due to the ability to flexibly extend the KG, the robot needs to deal with skills and problems when they arrive, which is a challenge to the complex behaviour, especially to IM. Due to limited resources, we seek a simple knowledge graph representation. Currently, the robot can only draw conclusions about its hierarchical relationships with skills and the Sensor and Actuator Space (SAS). However, recent work shows good progress in building rich socio-physical models for service robots, such as SOMA [5]. Knowledge graphs and their hierarchy can accelerate learning, as studied in Curriculum Learning (CL), which is an ML technique that conducts training in a meaningful order, from easier to more complex tasks [24][43]. In tinyDSM, the millirobot uses the generic KG to (a) infer the fulfilment of the sensorimotor abilities required for developing particular skills and (b) infer higher-order skills that may become available for development as it evolves. To represent semantic knowledge, we encode information through types of entities related to skill and the SAS. Let O be a set of skills with oi∈Oo_i∈ O, is a set of sensors with si∈s_i∈ and is a set of actuators with cj∈c_j∈ , as defined by the SAS. The set of entity labels (classes/types) follows with P=O∪P=O\,∪\, \,∪\, . Formally, the KG is a directed acyclic graph K=E,U,P,τK=\E,U,P,τ\, where E is a set of entities, U⊆(n,m)|(n,m)∈E×E∧n≠mU \(n,m)\,|\,(n,m)∈ E× E n≠ m\ is the set of directed edges and τ:E→Pτ:E→ P is a bijective function mapping labels to entities. A directed edge en→eme_n→ e_m in K between the SAS and skill entities indicates that a sensor/actuator associated with em∈△e_m∈ needs to be available to the robots physical body before it can start executing the skill associated en∈Oe_n∈ O. Further, a directed edge en→eme_n→ e_m in K indicates that eme_m should be learned before ene_n. Again, a skill is deemed ”learned” if f(oi)≥fthresholdf(o_i)≥ f_threshold. The set of masterable skills MOM_O at time t includes (a) skills that have been learned and (b) skills that are ready to be learned. Note that, in this work, the KG is fixed at runtime, but the framework is designed to support over-the-air updates to its the structure and parameters as new capabilities and sensorimotor knowledge become available. Fig. 3: (a) Generic KG of skills and SAS; (b) Millirobot embodiment defines available sensorimotor capabilities. Skills become masterable once their dependencies are satisfied (e.g., ATOMIC MOVE before MOVE). I-C Intrinsic Motivation We model the complex behavior of our millirobot based on its intrinsic motivation. The design aims to generate its own objectives and to pursue these autonomously, not because they are explicitly given in a strict sequence, but because they appear interesting due to various factors, driven by inner drives and “curiosity” [3, 28]. According to the taxonomy of IM models proposed by Oudeyer and Kaplan [31], our approach aligns most closely with the competence-based models. The millirobot is driven by its desire to improve its skills through a set of self-generated goals instead of primarily seeking novel or surprising stimuli, as often seen in knowledge-based models. We model the IM m(o,t)m(o,t) of an agent to pursue a masterable skill with o∈MOo∈ M_O as the product of three factors with m(o,t)=novelty(o,t)⋅progress(o,t)⋅difficulty(o,t)m(o,t)=novelty(o,t)·progress(o,t)·difficulty(o,t)\\ The complete formulation is detailed in Appendix -A. In the context of our work, the robot must be capable of continuous development without stagnation, ensuring it does not get stuck in specific scenarios. The novelty factor reflects how unfamiliar a skill is to the agent. It motivates the robot to explore and discover all skills available in its KG), ensuring that no skills are ignored or excluded from learning permanently. The progress factor, primarily driven by the fitness assessment, encourages consistent improvement (fast improvements boost motivation to continue learning) while enabling the millirobot to respond adaptively to regressions. The difficulty factor refers to the complexity of mastering a skill and ensures a balanced exploration, while more complex skills are prioritized simpler ones are still in focus periodically and never neglected completely. With this approach, the millirobot should ultimately be capable of developing all of its skills independently and without a final aim. Addressing the memory and runtime constraints, we seek for a lightweight implementation of the IM, computed using only minimal historical data. I-D Developmental Process Our proposed tinyDSM, is designed to minimize hard-wired skills while encouraging the open-ended development of new skills. The key is to encode a minimal set of general knowledge (see Fig. 1) that serves as a foundational starting point for the system. As development unfolds, it learns system-specific dependencies, like sensorimotor mapping skills, monitors them and adapts to new unseen conditions. It is designed to operate within the resource constraints of a low-energy microcontroller with only a few hundred kilobytes of RAM. Fig. 4: Overview of the tinyDSM developmental flow: the agent queries the KG to form a skill pool MOM_O, selects skills via intrinsic motivation, and uses fitness from sensor-based reasoning to drive skill learning and curriculum progression. tinyDSM is outlined in Fig. 4 and operates in discrete time steps. Beginning at time step t0t_0 the millirobot queries its KG using its available sensor and actuator set to generate a set of masterable skills, referred to as the Skill Pool MOM_O. If the SAS is not complete, say the millirobot has no motors, it cannot move at all and thus cannot resolve the respective motion related skills, and its skill pool remains empty. However, out of this set, the skill with the highest motivation at time t is loaded for learning. The learner generates a sequence of action commands AtA_t. The SAS translates them to robot specific actuator commands _t leading to physical action in the environment. The robot explores the effects by reading its sensors +1 _t+1. The set of reasoners monitors all state transitions Tt+1T_t+1 and compares the observed readings to expected assumptions. Based on that, the fitness Ft+1F_t+1 is calculated, acting as a reward signal for the learner to refine future actions. Based on Tt=1T_t=1 and Ft+1F_t+1 the algorithm performs its updates and reports the skills state Xt+1X_t+1 to tinyDSM. If fthresholdf_threshold is reached, the skill transitions from active learning to exploitation. In the next discrete time step, the agent re-queries the KG and updates the skill pool MOM_O. At this stage, various scenarios may arise that affect the course of future development: (a) If a skill is deemed learned, it may unlock more complex skills (Curriculum Learning). (b) If the KG was extended it may discover new skills. (c) If the robot undergoes physical modifications, such as adding a new sensor, its developmental space may grow. (d) If nothing of the above applies, it continues developing existing skills. The curriculum factor in the developmental process directs the agent toward valuable search spaces, maximizing learning efficiency and accelerating progress. In contrast, the intrinsic motivation factor enables autonomous exploration, mirroring adaptive behavior observed in humans and animals. This combination allows the system to rapidly converge to a better local optimum while continuously monitoring and adapting its performance. The agent achieves a structured yet adaptive learning progression by continuously refining its internal representation of learnable skills. This self-reinforcing loop, where knowledge expansion is contingent on prior mastery and environmental modifications, mimics principles observed in biological cognitive development, enhancing both the learning process’s autonomy and scalability. Learning routine (skill optimizer)— Once tinyDSM has selected a skill for development, a lightweight learning routine optimizes the policy of the skill under strict resource constraints. We are aware of the complexity of optimizing hyperparameters and formulating generalizable rewards, which in most applications require specific system-specific information to converge quickly. To address this challenge, we use small models with limited completeness and a minimalist reward design in a normalized 0-1 format. Curriculum learning and knowledge-based reasoner are used to guide the learning process to converge to local optima efficiently. tinyDSM monitors (fitness reasoner) all actions during learning and execution. This enables a transparent learning process and facilitates traceability and validation during execution. We distinguish between the developmental process implemented by tinyDSM (higher-order control logic) and the learning routine (the learner). tinyDSM acts a meta-controller that decides which skill should be learned, when learning should start or stop, and how the skills are organized through the knowledge graph and curriculum. In contrast, the learner is a task-local optimizer that determines how a selected skill is acquired by adjusting its control policy from experience. Due to resource constraints, we apply simulated annealing (SA) [16] for the learner. But any of the well established RL algorithms like Deep Q-Networks (DQN) [23] and Deep Deterministic Policy Gradient (DDPG) [20] are viable candidates. Genetic algorithms also show promise for policy optimization, as evolutionary approaches complement gradient-based methods in overcoming challenges like deceptive local optima and sparse rewards [40, 41]. IV Experimental Setup The tinyDSM framework is a low-level implementation in C++ that includes several modules, such as memory management, communication (SAS), skill, scheduler, learner, and agent, among others. Together with an extensive set of low-level libraries that are highly optimized for embedded devices, the framework enables the implementation of various skills and the integration of different learning algorithms, thus enabling efficient and flexible studies of developmental mechanisms in low-resource scenarios. In this work, we analyze the memory consumption categorized according to the minimal a priori knowledge and show the runtime performance of the framework without detailing design or implementation aspects. However, the setup for this study is depicted in Fig. 5, where the millirobot is placed in a controlled environment. A host device (desktop computer), captures the translation information using a camera and a fiducial marker system based on the ArUco library [13]. At this stage, embedded sensors for precise positioning are not yet integrated into the robot, so the host device transmits the sensed pose via Bluetooth. This limits autonomy and restricts experiments to controlled environments. Otherwise, this has no significant impact since there is no major delay in communication, and DSM works with any type of position sensor through the generic modelling of the skills, assuming the sensors provide the same types of values. Regarding resources, the communication overhead is about the same as an onboard sensor reading, if not slightly lower. In future work, we will integrate onboard localization to improve autonomy and scalability. The lightweight millirobot (150 g150\,g), measuring 40 40\,×30 m230\,m^2, is 3D-printed and features four wheels, two of which are motor-driven. The tinyDSM runs on an RP2040 (Raspberry Pi Pico [33]) 32 bit32\,bit microcontroller which has tight limited resources as listed in Table I. TABLE I: RP2040 Specifications. Processor FPU HW Integer Divider Frequency SRAM ARM Cortex-M0+ NO YES 133 MHz133\,MHz 264 k264\,k For the experiments, we model five motion-related skills. We group them into a foundational ATOMIC\ MOVE skill and a more specialized MOVE skill that builds upon it. These skills enable the millirobot to theoretically follow any geometric pattern, as we demonstrate with the MOVE\,SQUARE skill. As a prospect of a more complex extension, this could also be a navigation path generated by a corresponding NAVIGATION skill. We investigate how a newborn millirobot agent discovers these skills based on its physical body and how it develops and adapts them. The skills we propose are hierarchical, meaning that the agent must first master less complex skills before tackling more advanced ones. We examine in detail the intrinsically motivated behavior of the millirobot, which drives its development and must be able to react flexibly and effectively to a variety of unknown situations without stagnating. A key challenge ensuring the long-term maintenance of skills without neglecting the acquisition of new ones. Additionally, we evaluate the fitness assessments and demonstrate how these can aid in adapting to system degradation or changes in the environment by adding weight to the robot while operating. In addition to the real-world experiments, we conducted a simulation-based analysis to evaluate learning behavior under varying intrinsic motivation settings (Sec. IV-E). Fig. 5: The millirobot is 3D-printed and features four wheels, two of which are motor-driven. The translation information is captured on a host device using a camera and a fiducial marker system and transmitted to the embedded RP2040, which operates the tinyDSM. IV-A SAS The millirobots physical model is defined with SASrobot=(,) SAS_robot=( , ) and organized in its body graph as illustrated in Fig. 3(b). It controls two motors =cm1,cm2 =\c_m_1,c_m_2\. The corresponding motor command takes three parameters, the time of activation t, the rotational force f and the duration τ how long the force is applied: cm1(t,f,τ)c_m_1(t,f,τ) and cm2(t,f,τ)c_m_2(t,f,τ). It reads one sensor sp∈s_p∈ that provides the pose with cartesian coordinate information sp(t)→[x,y,ψ]s_p(t)→[x,y,ψ] at time t, with x and y denoting the horizontal and vertical coordinate in the x- and y-dimensions, respectively, and ψ denoting the yaw angle. The fundamental concept is that these interfaces offer an efficient abstraction of the robot-specific sensor readings and actuator commands, enabling higher-level, studies of generic methods without being constrained by robot-specific implementations. For example, motor control, whether managed e.g. through PWM, needs to be modelled for the specific robotic system. It allows the system to function without needing to know the specific actuators or sensors being used, it can simply rely on the defined interfaces. IV-B Kinematic Reasoner For motions, we refer to the basic kinematic and dynamic properties of a system, with kinematics describing the relationship between coordinates in motion space. With the dynamics correlating to the torque and force in each wheel. When the wheels touch the ground, these forces act indirectly on the overall system and thus cause it to move. The resulting spatial movement is determined by the change in position over time t using an inverse kinematic reasoner with Δx=xt−xt+1,Δy=yt−yt+1, x=x_t-x_t+1,\; y=y_t-y_t+1, (2) Δψ=(ψ0,ψt) ψ=wrap( _0, _t) With Δx x being the horizontal change in distance, Δy y is the vertical change, while Δψ ψ denotes the difference in orientation, wrapped to a specified range of [−180∘,180∘][-180 ,180 ] with (ψt,ψt+1)wrap( _t, _t+1). The reasoner utilities this information to infer (a) linear and (b) angular motion patterns. A linear movement is defined with the euclidean distance d=Δx2+Δy2≠0d= x^2+ y^2≠ 0 and Δψ=0 ψ=0. While Δψ≠0 ψ≠ 0 and d=0d=0 indicate a pure angular motion pattern. This general knowledge pertains to two-dimensional space and can be used to infer motion for any moving object, in particular to various robots. Reasoners play a crucial role in knowledge-based systems, enabling logical inference and decision-making from incomplete or structured information. In our design, the kinematic reasoner is particularly characterized, as it adds to the basis of the minimal set of initial knowledge required for the robot to develop. Moreover, an extension to n-dimensional space can be defined in an analogous manner. IV-C Skills At the foundational level of the knowledge graph, the ATMOIC\ MOVE skill type models basic linear and angular motion patterns. Building upon this, the MOVE skill type models movements over specific ranges, such as travelling specific linear distances or rotating by set specific angles. The most complex skill MOVE\ SQUARE uses these lower-level skills to execute a series of movements that outline the shape of a rectangle. forest []ATOMIC\ MOVE\ [LINEAR]—oAMLo_AML: It is designed to enable the millirobot to drive in a straight line with as little rotation as possible with no specific distance. The specification DSAML=(T,F)D_S_AML=(T,F) includes the desired transformation information T with set of sensor reading S=spS=\s_p\ where sp∈Ts_p∈ T. As defined by the SAS, sp(t)→[x,y,ψ]s_p(t)→[x,y,ψ] provides the cartesian coordinate information at time t. The vectors of the fitness F follow with: t→= t= (max(Δx,20 m)0 m0 °) pmatrix ( x,$20\,m$)\\ $0\,m$\\ $0\, $ pmatrix o→= o= (ΔxΔyΔψ) pmatrix x\\ y\\ ψ pmatrix r→= r= (t0t0/410 °) pmatrixt_0\\ t_0/4\\ $10\, $ pmatrix with the target vector t→ t, the observations o→ o and the permissible deviations r→ r. The observation vector o→ o captures the sensor readings based on T. These are then processed by the kinematic reasoner (see 2) to determine the relative movement of the millirobot. During learning the agent utilizes its algorithm that generates a sequence of motor commands A=⟨cm1,cm2⟩A= c_m_1,c_m_2 with cm1(t)→(f1,τ1)c_m_1(t)→(f_1, _1) and cm2(t)→(f2,τ2)c_m_2(t)→(f_2, _2). The duration τ, how long the force is applied, is set with a constant value τ1=Cτ _1=C_τ and τ2=Cτ _2=C_τ to simplify learning, as we search for any straight-line movement regardless of the distance traveled. Since no specific distance is required, we need to ensure the agent does not learn action commands that are too small. To achieve this, we set a minimum distance of the target t→0 t_0 to 20 m20\,m for the calculation of the fitness score. It still ensures that the motion patterns being explored are independent of the distance traveled. Lateral or rotational movements are not desired. Thus, the target for both is zero. For the same reason as for t→ t the members of r→ r need to be independent of the distance traveled, thus leaving only forces f1f_1 and f2f_2 to be determined by the learner. []ATOMIC\ MOVE\ [ANGULAR]—oAMAo_AMA: It is analogously designed to its linear counterpart oAMLo_AML. It enables the millirobot to rotate in place, without specific angles and with as little movement in the directions Δx x and Δy y. The specification is DSAMA=(T,F)D_S_AMA=(T,F) with S=spS=\s_p\ where sp∈Ts_p∈ T and F with: t→= t= (0 m0 max(Δψ,20 °)) pmatrix$0\,m$\\ $0\,m$\\ max( ψ,$20\, $) pmatrix o→= o= (ΔxΔyΔψ) pmatrix x\\ y\\ ψ pmatrix r→= r= (10 m10 mmt2) pmatrix$10\,m$\\ $10\,m$\\ t_2 pmatrix The actuator commands are specified with A=⟨cm1,cm2⟩A= c_m_1,c_m_2 . As no specific angle is given, τ is also set constant with τ1=Ct _1=C_t and τ2=Ct _2=C_t. As above, the learner has to find solutions to f1f_1 and f2f_2. []MOVE\ [LINEAR]—oMLo_ML: This skill builds upon oAMLo_AML to enable the millirobot to drive in a straight line and stop at a specific distance relative to the starting position. The specification is DSML=(T,F)D_S_ML=(T,F) with S=spS=\s_p\ where sp∈Ts_p∈ T. Since only the driven distance matters for this skill, the vectors of the fitness F are straightforward: t→=(dl) t= pmatrixd_l pmatrix o→=(Δx) o= pmatrix x pmatrix r→=(dl2) r= pmatrix d_l2 pmatrix With dld_l being the specific distance the robot has to travel. The observations for o→ o are calculated by the kinematic reasoner (see 2) based on the T. The learning algorithm generates A=⟨cm1,cm2⟩A= c_m_1,c_m_2 with cm1(t)→(f1,τ1)c_m_1(t)→(f_1, _1) and cm2(t)→(f2,τ2)c_m_2(t)→(f_2, _2). Since the oAMLo_AML already provides f1f_1 and f2f_2 for driving in a straight line, the learner solely has to provide the mapping τ(dl)τ(d_l) where τ=τ1=τ2τ= _1= _2. []MOVE\ [ANGUALAR]—oMAo_MA: Again, it is analogously designed to its linear counterpart oMLo_ML and builds upon oAMAo_AMA to enable rotation by a specific angular distance dad_a. The specification is DSMA=(T,F)D_S_MA=(T,F) with S=spS=\s_p\ where sp∈Ts_p∈ T. t→=(da) t= pmatrixd_a pmatrix o→=(Δψ) o= pmatrix ψ pmatrix r→=(da2) r= pmatrix d_a2 pmatrix With dad_a being the specific angle the robot has to rotate. The actuator commands are specified with A=⟨cm1,cm2⟩A= c_m_1,c_m_2 . Since the oAMAo_AMA already provides f1f_1 and f2f_2, the RL algorithm searches for the mapping τ(da)τ(d_a) where τ=τ1=τ2τ= _1= _2. MOVE\ SQUARE—oMSo_MS: This skill is a simplified type of skill, as it lacks a learning routine and has A already predefined. Once the skill is invoked, it simply applies A. It is used to evaluate all previously learned skills in a compound. The ”task” is designed to trace a geometric pattern in a square, which is achieved by a sequence of linear and angular movements. The specification is DSMA=(T,F)D_S_MA=(T,F) with S=spS=\s_p\ where sp∈Ts_p∈ T. The fitness of the geometric pattern is evaluated based on the absolute differences between the global x- and y-coordinates of the robots starting and ending points upon completing the task with F: t→= t= (0 m0 m) pmatrix$0\,m$\\ $0\,m$\\ pmatrix o→= o= (|Δx||Δy|) pmatrix| x|\\ | y|\\ pmatrix r→= r= (150 m) pmatrix$150\,m$\\ pmatrix The predefined sequence of action commands A follow with: (⟨oML(50 m),oMA(90 °)⟩)i=14 (\; o_ML($50\,m$),\;o_MA($90\, $) \; )_i=1^4 (3) In this context, oMLo_ML signifies a linear displacement of 50 m50\,m, while oMAo_MA indicates a rotation of 90 °90\, , with these movements repeated as necessary to complete the rectangular pattern. IV-D Parameterization We configure all skills with fthreshold=0.95f_threshold=0.95 and the intrinsic motivation as listed in Table I. TABLE I: IM Parameterization. Ninit=50.0N_init=50.0 Nmax=100.0N_max=100.0 β=0.1β=0.1 γ=0.003γ=0.003 pscale=80.0p_scale=80.0 poffset=20.0p_offset=20.0 IV-E Simulation Based Analysis To investigate the developmental dynamics in a controlled and repeatable evaluation, we implemented a physics-based simulation environment in Python using pygame. The setup mirrors the SAS of the real millirobot, which ensures direct transferability between simulation and physical experiments. We evaluated three learning algorithms: • Simulated Annealing (SA) - used on the real robot, • Q-learning - a Q-Table RL, • Random - and a uniform random action selection (as a lower-bound reference). We operated all learners under identical intrinsic motivation, fitness threshold, and knowledge-graph structure. Each configuration was evaluated over multiple independent runs with different random seeds. Performance is evaluated with the mean skill fitness, defined as the average fitness across the five motion-related skills. This continuous metric provides a compact and sensitive measure of general developmental behavioural abilities. To characterize the internal dynamics of the intrinsic motivation, we define two metrics: (i) selection entropy NmaxN_ , that measures how evenly skills are selected, and (i) maximum neglect H, that measures how long any skill remains unselected. Both metrics are computed from per-skill selection statistics and recency counters (formal definitions are provided in Appendix -A). We evaluated six intrinsic motivation parameterizations defined in Table I. These configurations systematically bias the scheduler towards exploration (high_explore), exploitation (high_exploit), higher novelty ranges (high_Nlimit), earlier mastery (lower_fthr), or stronger convergence after the threshold (high_postslope). TABLE I: IM parameter configurations used in the simulation experiments. IM NinitN_init NlimitN_limit β γ fthrf_thr ppostp_post baseline 50 100 0.10 0.003 0.95 20 high_explore 50 100 0.10 0.010 0.95 20 high_exploit 50 100 0.20 0.003 0.95 20 high_Nlimit 50 150 0.10 0.003 0.95 20 lower_fthr 50 100 0.10 0.003 0.90 20 high_postslope 50 100 0.10 0.003 0.95 35 To compare IM configurations quantitatively, we define an IM Score which combines learning performance with IM scheduling stability. For each run, we compute the time-averaged mean fitness f¯ f, the time-averaged logarithmic maximum neglect N¯max N_ , and the time-averaged selection entropy H¯ H. These are aggregated per settings and combined as IMScore=f¯−λN¯max+μH¯,IM\;Score= f-λ\, N_ +μ\, H, (4) with λ=0.25λ=0.25 that penalizes the long-term skill neglect and μ=0.05μ=0.05 that rewards the exploration diversity. (a) (b) Fig. 6: V-A: (a) The intrinsic motivation m of the agent for each skill throughout its development, observed over a period of 12 min12\, . The gray segments highlight significant events that we will discuss in the text. (b) The corresponding fitness f for each skill. The blue diamond markers indicate the discovery of new skills. The experience on the right axis of both figures reflects the overall progress of the agent, as defined in Appendix -B. V Experimental Results First, in V-A, we demonstrate how the millirobot efficiently develops its skills in a reasonable timeframe through its flexible and adaptive behavior. We analyze in detail various factors that contribute to its intrinsic motivation and how these factors influence its actions. Next, in V-B, we explore how the system reacts to environmental changes, such as when the robot is loaded with weight, and how it adapts efficiently in a short time period. In V-C, we discuss the fitness evaluation in detail, illustrating how it can be applied to various higher-level skills in a scalable manner and the valuable insights we can gain from this process. In V-D, we outline the resources required by our framework, as our tinyDSM must operate effectively in targets with limited resources. Additionally, in V-E, we evaluate the learning behavior and intrinsic motivation dynamics in a simulated environment. V-A Development The millirobot is placed in a controlled environment without developed skills and must learn them from scratch. It starts with its initial prior knowledge as described in Fig. 1. First, the agent queries its KG using its available sensor and actuator set and generates the skill pool MOM_O. In Fig. 6b the blue diamond markers indicate when the agents skill pool is expanded. Shortly after the start, around 100 ms100\,ms, only oAMAo_AMA and oAMLo_AML are available, as they do not have dependencies other than on the SAS. This query process occurs at each discrete time step, which may lead to an expansion of the skill pool when lower-order skills reach their fthresholdf_threshold and higher-order skills are available in the KG. If a new skill is discovered the agent may start developing it. This behavior demonstrates the curriculum learning element of tinyDSM, which guides the agent during initial learning phases to explore valuable search spaces. Once a skill is discovered, the agents behaviour is driven by its intrinsic motivation, illustrated in Fig. 6a. It is calculated based on three factors (I-C): novelty, progress, and difficulty. Each of these factors influences the agent’s behavior depending on its current state and the experiences gained during the development process. In Fig. 6 eight relevant segments are highlighted and labeled (A) to (H). In segment (A), the motivation for both types of atomic motions increases sharply due to the novelty of these skills. They are being discovered for the first time, with no prior experience available. The millirobot randomly selects one skill since both motivations are equal. The agent starts by exploring oAMAo_AMA. The skill oAMLo_AML also receives sporadic attention, although faster progress is made with the angular pattern, as indicated by increases in fitness (Fig. 6b (A)). Between segments (A) and (B), the millirobot catches up with the progress of oAMLo_AML. In (B), the development of both skills occurs alternately, as the growth in fitness shows significant similarities. Other experiments have shown that motivation sometimes exhibits a ”latching” behaviour, focusing on a specific skill for an extended period before switching to the other. In such cases, the fitness of the focused skill increases significantly faster than the other skill. However, during (B), the motivation for both skills decreases, even though fitness continues to grow. This decline in motivation is attributed to the novelty and fitness, as the agent becomes more competent and ”bored”. In (C), after around 1.8 min1.8\, , the millirobot successfully reached the fthresholdf_threshold for oAMAo_AMA, resulting in experience (defined in Appendix -B) increasing to 11. This leads to the discovery of oMAo_MA and an expansion of the skill pool. Achieving this, took only about ∼50 50 interactions with the physical environment, which is fairly efficient. oMAo_MA is novel, and the agent is motivated to explore it. That skill is also more complex, as it needs the oAMAo_AMA for its activities. This is modeled with the higher hierarchical order in the KG, which results in a greater difficulty factor that increases motivation further. In section (D), the agent focuses only on developing oMLo_ML. While its fitness grows and it progresses with that skill, the motivation decreases, due to the decreasing novelty. This is the same pattern as in segment (B) and will continue to appear throughout development. What is new in this phase is that the agent becomes more interested in rediscovering the two atomic skills, so the motivation for those slightly increases again. However, the agent is slightly more motivated to pursue oAMLo_AML since it considers itself more experienced with oAMAo_AMA. In segment (E), the agent faces a new situation as it progresses with oMAo_MA and reaches fthresholdf_threshold, driving its experience to 22. It briefly keeps its focus on angular motions before facing oAMLo_AML again. This is delayed due to the higher difficulty factor. However, the agent reaches the oAMLo_AML threshold in the next interaction, increasing its experience to 33. That leads to the discovery of the oMLo_ML. Shortly after a few interactions, the agent focuses on this newly discovered skill, repeating the pattern observed in (C) and (F). In segment (F), omso_ms is discovered, and the millirobot is motivated to drive on a square path, exploiting all previously learned skills in a sequence. However, the agent focuses on navigating along a square for the rest of the experiment. This behavior aligns with the intended objective, as more complex skills typically offer greater utility and adaptability than simpler ones. Even though it focuses on more complex skills, it does not entirely ignore simper atomic moves, visible in segment (H). In a brief segment (G), the robot successfully executed some high-quality squares indicated by toggling the experience to 55. Note that the oMSo_MS has no learner, so the agent cannot improve it directly. We discuss this aspect in more detail in experiment V-C. Finally, the fitness is monitored for each interaction and skill, with the agent immediately recognizing environmental changes. As a result, the agent must be able to adapt to these changes. This challenge will be analyzed in experiment V-B. In summary, in this experiment we demonstrated that the millirobot can develop skills efficiently and without stagnation in a relatively short time utilizing tinyDSM mechanisms and driving the development by combining intrinsic motivation and curriculum-based learning. Notably, the millirobot learned basic movements in only ∼4.5 min $4.5\, $ and could, in principle, navigate effectively with the respective knowledge acquired. V-B Adaption Next, we investigate how the millirobot reacts to environmental changes. We begin by placing the robot in a controlled environment without developed skills. Fig. 7 shows the progress of the robot. After roughly ∼6.5 min $6.5\, $, we load it with additional weight. This is indicated by the start of the gray shaded segment (W). This alteration in its physical properties invalidates the previously learned movement parameters, as the millirobot now reacts differently to the same motor commands. Fig. 7: V-B: The upper graph shows the intrinsic motivation m of the agent. The lower graph shows the corresponding fitness f for the respective abilities. The dashed line illustrates the agents experience. Once the development has progressed and the robot has experience, it is loaded with additional weight of 380 g380\,g. This is roughly after 6.5 min6.5\, , which is indicated by the orange diamond marker. This results in a significant drop in fitness values, which motivates the agent to relearn the affected skills during (W). Shortly after this weight change, we observe a significant drop in the fitness values of both oMAo_MA and oMLo_ML. Additionally, the agents experience decreases as its skills drop below the fitness threshold. Note, the weight change only affects the learned duration of the motor commands τ and does not impact the ratios between f1f_1 and f2f_2. Consequently, the atomic motion skills oAMAo_AMA and oAMLo_AML remain unaffected. However, the decrease in fitness results in a sharp increase in motivation, primarily caused by the progress factor that drives the robots interest in relearning these skills. During (W), which lasts for ∼9 min $9\, $, the agent successfully relearns its skills, as evidenced by increasing fitness scores and improvement in experience. In summary, through the use of tinyDSM, the agent continuously monitors all actions, even if the skills are sufficiently developed. It can detect deviations from learned actions through fitness computations. Ultimately, this allows the agent to respond effectively with its behaviour to new and unseen situations, adapting its skills to different conditions. V-C Fitness Assessment Fig. 8 illustrates the fitness of oMAo_MA and oMLo_ML during learning in two distinct progress stages. Movements with less learning progress (0.7>f<0.950.7>f<0.95) show a very high variance and deviate significantly from the target, while more developed movements (f>=0.95f>=0.95) are much closer to the target and show a significant reduction in variance. However, some variance remains even at high fitness values. We attribute this to the physical properties of our millirobot, which is far from ideal. (a) (b) Fig. 8: V-C: Fitness evolution of angular (oMAo_MA, red) and linear (oMLo_ML, green) motion skills during learning. Observations o→ o (left axis) are compared to targets t→ t, with fitness f shown on the right. Two learning stages (0.7<f<0.950.7<f<0.95 and f≥0.95f≥ 0.95) show reduced variance as skills develop. During the experiments, we observed that the robot sometimes reacts differently to the same motor commands. Particularly noteworthy is that, it experiences a “push” in the braking phase before it stops. This phenomenon is due to the unpredictable stalling of the motors gearbox when turning off. That results in an extended movement and can cause the robot to overshoot the target. This is also evident in Fig. 9, where, on average, all squares remain clearly “open” and miss the target on the left. Only a few good outliers pass it on to the right. The trajectory deviates from a perfect square shape and does not close exactly at the endpoint, observable in the significant fluctuations in the fitness calculation of the depending motion skills. However, we do not aim for a highly optimized control system but rather want to explore a general approach to learning movement. An alternative general approach would involve discovering a series of small movements. A navigation skill could generate the path and correct errors, enabling more precise navigation. The perspective on error considerations and error tolerance of a particular function is always challenging. However, the error propagates over the sequence of 88 movements when following a square path. Thus, we argue that the accumulated navigation error, averaging 7.5 m7.5\,m, is sufficiently small to ensure acceptable navigation, especially when considering the footprint size of the robot and the non-idealities of the motor hardware. In summary, we demonstrate that our fitness calculation scales properly and seamlessly applies to more complex skills. The hierarchical structure is also well reflected in the fitness calculation of the square. Errors at lower hierarchical levels directly affect higher levels. By recognizing these errors at different levels, we can correct deviations in a targeted manner. This leads to a significant increase in efficiency, as only the skills that are actually affected need to be re-learned, as demonstrated with the adaptation experiment. Fig. 9: V-C: The Square motion skill oMSo_MS projected with the first segment horizontal. The robot starts at the red point and executes alternating 50 m50\,m linear and 90 °90\, angular moves to form a square. Left: f≥85f≥ 85; right: f≥93f≥ 93. V-D Resources In terms of resources, we consider the memory and execution time of the whole runtime system during the specific scenarios in our experiments. Since tinyDSM was designed for flexible use, we aim to reflect this flexibility in its implementation. TABLE IV: Static memory usage per module, including total allocation. Memory Module tinyDSM Usage Agent 280 B280\,B 5.7%5.7\% Skills 488 B488\,B 10.0%10.0\% Learners 2544 B2544\,B 52.1%52.1\% SAS 840 B840\,B 17.2%17.2\% Scheduler 728 B728\,B 15.0%15.0\% M 4017 B4017\,B −- Total 8897 B8897\,B 100%100\% tinyDSM is designed to only allocate memory during startup and skill creation and never during the normal execution cycle. This makes the memory consumption very predictable, minimizes bookkeeping for the memory manager and reduces the possibility of out-of-memory errors. Table IV illustrates the static memory consumption. It details the memory usage for each module. The five skills tested in the experiments require 488 B488\,B, while the communication module (SAS) uses 720 B720\,B and the scheduler (IM) requires 840 B840\,B. However, these compact modules represent only a small part, with learners consuming a significant 52%52\% of the total memory. Learning algorithms, particularly neural networks are memory intensive. Therefore, we focused on a memory-efficient implementation when designing the framework to ensure there is enough space for these algorithms. In this particular case, the numbers for the learners are inflated by a factor of ≈4≈ 4. This is because of internal memory fragmentation caused by the buddy system having a minimum allocation size of 64 B64\,B and therefore being unsuited for many very small allocations. Also the use of dynamic arrays in the learners for arrays which are in essence constant in length and only 1 to 2 entries long causes a significant increase in memory consumption in the learners. As an outlook, we estimate how many skills could be located on the RP2040 in a fictive scenario. We scale the framework to utilize the full 250 kB250\,kB(16 kB16\,kB reserved for pico SDK) available memory of the pico. We assume one of the move skills for the workload calculation, as this is the most complex and requires the most memory, say the linear move skill. Based on that, our calculations show that 9696 instances of that skill would fit in the pico. This means that the agent would learn 9696 separate linear movements, which is not a useful task but demonstrates tinyDSMs potential. In terms of resource balancing, let’s consider a successfully developed skill (threshold reached). In this case, the framework could release the memory allocated for the particular learning algorithm. This leads to a dynamic adjustment of resources, allowing the millirobot to balance its workload according to its needs. For instance, in segment (H) (see Fig. 6b), no learners are required since all skills have been learned, and oMSo_MS has no learner. The memory of 2544 B2544\,B (Table IV) can be freed. The robot can still explore its environment and search for new interesting skills, but it requires less energy than in its development phase. This property is ideal for operating in low-resource environments. The tinyDSM uses a dynamic Memory Management (M), which is implemented using a custom buddy memory allocator. Where each module manages its own memory region, sometimes using its own allocation techniques. One example is the use of a simple memory arena or bump allocator in the skills module, which almost completely removes internal memory fragmentation within this module. This principle could also be applied to the other modules, reducing memory usage even further. As for the other resources, the execution time of a single learning step varies from 10 ms to 30 ms10\,ms30\,ms depending on the complexity of the skill. In terms of weight, the millirobot itself weighs 150 g150\,g and the additional weight used in the experiments is 380 g380\,g. During all our experiments, the robot operated for about ∼45 min $45\, $ on a battery with a capacity of 250 mA h250\,mA\,h. V-E Simulation-Based Analysis V-E1 Skill Acquisition Across Learning Algorithms Both Q-learning and SA rapidly acquire the full skill set, achieving near-optimal performance, whereas the random policy fails to develop meaningful behaviour (Fig. 10). Fig. 10: Average fitness f¯ f for for different learning algorithms (mean with min–max across runs). The Q-learning converges fastest, followed closely by SA. This indicates that the intrinsic motivation paired with the curriculum mechanism based on the KG in tinyDSM can effectively guide both learners toward the relevant search spaces. The random baseline stagnates far below the threshold for meaningful skill acquisition. This is cause more complex skills, such as higher-order move skills, are the primary bottlenecks. Random policies never succeed in these skills, whereas SA and Q-learning both master them. V-E2 Intrinsic Motivation Dynamics Fig. 11 shows the corresponding IM analysis. High exploration (high_explore, large γ) leads to high selection entropy but also extreme maximum neglect, indicating that some skills are ignored for long periods. High exploitation (high_exploit, large β) maintains low neglect but slightly reduces coverage. The baseline configuration maintains both high entropy (≈0.83≈ 0.83) and low maximum neglect (≈1.37≈ 1.37), indicating a balanced and stable development process. Fig. 11: Intrinsic Motivation Dynamics. Top: maximum neglect Nmax(t)N_ (t). Bottom: selection entropy H(t)H(t). Table V shows the IM Score for each configuration. Since the score combines the learning quality (f¯ f) with scheduling stability (N¯max N_ and H¯ H), it directly mirrors the trade-off between fast skill acquisition and robust development behaviour. The baseline configuration achieves the highest IM rating, which is also evident in the visual representation in It incorporates fast, high-quality learning with stable and well-balanced skill scheduling. The high_exploit configuration reduces maximum neglect but also slightly lowers selection entropy, indicating an early focus on a narrow development schedule. The high_Nlimit maintains exploration diversity but allows for higher neglect by focusing too much on novelty-oriented exploration. The lower_fthr setting leads to early mastery but increases neglect, where some skills are effectively never selected again on the timescale of learning. On the other hand, high_postslope keeps mastered skills highly attractive, causing the scheduler to repeatedly revisit a few skills and thereby reducing the stability of the developmental schedule. Finally, the high_explore performs worst, as it focuses on extreme novelty, which causes the IM scheduler to neglect previously learned skills, resulting in long-term starvation of many skills despite high overall entropy. Table V summarizes the resulting scores. TABLE V: IM configuration comparison. Higher IM Score is better. IM f¯ f H¯ H N¯max N_ IM Score baseline 0.998 0.833 1.37 0.675 high_exploit 0.997 0.846 1.46 0.650 high_Nlimit 0.995 0.856 1.48 0.637 lower_fthr 0.997 0.829 1.51 0.628 high_postslope 0.999 0.865 1.48 0.626 high_explore 0.992 0.807 1.82 0.503 These results demonstrate that tinyDSM is highly sensitive to intrinsic motivation parameterization. Increasing the novelty destabilizes development through excessive neglect, whereas stronger exploitation accelerates early learning at the cost of reduced diversity. The baseline configuration offers the most robust development schedule. This indicates that IM scheduling plays a stability-critical role in shaping developmental dynamics. VI Conclusion In this work, we introduced tinyDSM, an advanced developmental skill method specifically designed for resource-constrained millirobots, bridging the domains of tiny robot learning and cognitive developmental robotics. tinyDSM bridges the domains of tiny robot learning and cognitive developmental robotics by enabling autonomous, open-ended skill development from minimal prior knowledge. It combines intrinsic motivation, fitness-based evaluation, and structured knowledge representation through a hierarchical knowledge graph and kinematic reasoning. We demonstrated that a millirobot with a volume of only 36 cm336\,cm^3, running on a Raspberry Pi Pico 32-bit microcontroller (RP2040), can progress from atomic motion patterns to complex geometric behaviours within 15 minutes, despite operating within just 9 kB9\,kB of memory. This is achieved through curriculum-based learning and an intrinsic motivation model balancing novelty, progress, and difficulty. To assess generality and robustness, we conducted a complementary simulation-based analysis. This allowed systematic evaluation across different learning algorithms and intrinsic motivation configurations. The results highlight that the developmental dynamics and stability of skill acquisition are highly sensitive to motivation scheduling and parameterization. We showed that the agent efficiently developed core motion skills, adapted to physical changes like added weight, and maintained performance through continuous fitness-based evaluation. The system shows robustness in maintaining previously acquired skills while pursuing new competencies, supporting the concept of lifelong learning. These results demonstrate the potential of flexible, self-directed robotic agents that can learn throughout their lifetime in dynamic environments with limited resources. Future work will explore richer sensing, higher-level planning, and on-device learning with lightweight neural networks, along with studies in more complex scenarios. References [1] J. Andreas, D. Klein, and S. Levine (2017) Modular Multitask Reinforcement Learning with Policy Sketches. arXiv:1611.01796 [cs]. External Links: 1611.01796 Cited by: §I. [2] M. Asada, K. Hosoda, Y. Kuniyoshi, H. Ishiguro, T. Inui, Y. Yoshikawa, M. Ogino, and C. Yoshida (2009) Cognitive Developmental Robotics: A Survey. IEEE Transactions on Autonomous Mental Development 1 (1), p. 12–34. External Links: ISSN 1943-0612, Document Cited by: §I, §I. [3] A. Baranes and P. Oudeyer (2013) Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots. Robotics and Autonomous Systems 61 (1), p. 49–73. External Links: 1301.4862, ISSN 09218890, Document Cited by: §I, §I-C. [4] A. G. Barto, S. Singh, and N. Chentanez (2004) Intrinsically Motivated Learning of Hierarchical Collections of Skills. Cited by: §I. [5] D. Beßler, D. Ler, R. Porzel, M. Pomarlan, A. Vyas, S. Höffner, S. Ffner, M. Beetz, R. Malaka, and J. Bateman (2021) Foundations of the Socio-Physical Model of Activities (SOMA) for Autonomous Robotic Agents. Formal Ontology in Information Systems, p. 159–174. External Links: Document Cited by: §I-B. [6] A. Cangelosi and M. Schlesinger (2018) From Babies to Robots: The Contribution of Developmental Robotics to Developmental Psychology. Child Development Perspectives 12 (3), p. 183–188. External Links: ISSN 1750-8606, Document Cited by: §I, §I. [7] G. Caprari (2003) Autonomous micro-robots : applications and limitations. Ph.D. Thesis, Lausanne, EPFL. External Links: Document Cited by: §I. [8] J. Chen, Y. Chen, X. Zhang, X. Du, K. Wang, and J. Wen (2018) Entity set expansion with semantic features of knowledge graphs. Journal of Web Semantics 52–53, p. 33–44. External Links: ISSN 15708268, Document Cited by: §I-B. [9] C. Colas, P. Fournier, O. Sigaud, M. Chetouani, and P. Oudeyer (2019) CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning. arXiv. External Links: 1810.06284, Document Cited by: §I. [10] S. D. de Rivaz, B. Goldberg, N. Doshi, K. Jayaram, J. Zhou, and R. J. Wood (2018) Inverted and vertical climbing of a quadrupedal microrobot using electroadhesion. Science Robotics 3 (25), p. eaau3038. External Links: Document Cited by: §I. [11] B. P. Duisterhof, S. Krishnan, J. J. Cruz, C. R. Banbury, W. Fu, A. Faust, G. C. H. E. de Croon, and V. Janapa Reddi (2021) Tiny Robot Learning (tinyRL) for Source Seeking on a Nano Quadcopter. In 2021 IEEE International Conference on Robotics and Automation (ICRA), p. 7242–7248. External Links: ISSN 2577-087X, Document Cited by: §I. [12] S. Forestier, R. Portelas, Y. Mollard, and P. Oudeyer (2022) Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning. Journal of Machine Learning Research 23 (152), p. 1–41. External Links: ISSN 1533-7928 Cited by: §I. [13] S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47 (6), p. 2280–2292. External Links: ISSN 0031-3203, Document Cited by: §IV. [14] R. Immonen and T. Hämäläinen (2022) Tiny Machine Learning for Resource-Constrained Microcontrollers. Journal of Sensors 2022 (1), p. 7437023. External Links: ISSN 1687-7268, Document Cited by: §I. [15] P. Khandelwal, S. Zhang, J. Sinapov, M. Leonetti, J. Thomason, F. Yang, I. Gori, M. Svetlik, P. Khante, V. Lifschitz, J. K. Aggarwal, R. Mooney, and P. Stone (2017) BWIBots: A platform for bridging the gap between AI and human–robot interaction research. The International Journal of Robotics Research 36 (5-7), p. 635–659. External Links: ISSN 0278-3649, Document Cited by: §I. [16] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi (1983) Optimization by Simulated Annealing. Science 220 (4598), p. 671–680. External Links: Document Cited by: §I-D. [17] J. E. Laird, K. R. Kinkade, S. Mohan, and J. Z. Xu Cognitive Robotics Using the Soar Cognitive Architecture. Cited by: §I. [18] M. T. Lê, P. Wolinski, and J. Arbel (2023) Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review. External Links: Document Cited by: §I. [19] H. Levesque and G. Lakemeyer (2008) Chapter 23 Cognitive Robotics. In Foundations of Artificial Intelligence, F. van Harmelen, V. Lifschitz, and B. Porter (Eds.), Handbook of Knowledge Representation, Vol. 3, p. 869–886. External Links: Document Cited by: §I. [20] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra (2019) Continuous control with deep reinforcement learning. arXiv:1509.02971. External Links: 1509.02971 Cited by: §I, §I-D. [21] J. Lin, L. Zhu, W. Chen, W. Wang, and S. Han (2023) Tiny Machine Learning: Progress and Futures [Feature]. IEEE Circuits and Systems Magazine 23 (3), p. 8–34. External Links: ISSN 1558-0830, Document Cited by: §I. [22] M. Lungarella, G. Metta, R. Pfeifer, and G. Sandini (2003) Developmental robotics: a survey. Connection Science 15 (4), p. 151–190. External Links: ISSN 0954-0091, Document Cited by: §I, §I. [23] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller (2013) Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602. External Links: 1312.5602 Cited by: §I, §I-D. [24] S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone (2020) Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey. ArXiv. Cited by: §I-B. [25] S. M. Neuman, B. Plancher, B. P. Duisterhof, S. Krishnan, C. Banbury, M. Mazumder, S. Prakash, J. Jabbour, A. Faust, G. C.H.E. de Croon, and V. J. Reddi (2022) Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robots. In 2022 IEEE 4th AICAS, p. 296–299. External Links: Document Cited by: §I, §I. [26] S. M. Nguyen, N. Duminy, A. Manoury, D. Duhaut, and C. Buche (2021) Robots Learn Increasingly Complex Tasks with Intrinsic Motivation and Automatic Curriculum Learning. KI - Künstliche Intelligenz 35 (1), p. 81–90. External Links: ISSN 1610-1987, Document Cited by: §I. [27] J. I. Olszewska, M. Barreto, J. Bermejo-Alonso, J. Carbonera, A. Chibani, S. Fiorini, P. Goncalves, M. Habib, A. Khamis, A. Olivares, E. P. de Freitas, E. Prestes, S. V. Ragavan, S. Redfield, R. Sanz, B. Spencer, and H. Li (2017) Ontology for autonomous robotics. In 2017 26th IEEE RO-MAN, p. 189–194. External Links: ISSN 1944-9437, Document Cited by: §I, §I. [28] P. Oudeyer, F. Kaplan, and V. V. Hafner (2007) Intrinsic Motivation Systems for Autonomous Mental Development. IEEE Transactions on Evolutionary Computation 11 (2), p. 265–286. External Links: ISSN 1089-778X, Document Cited by: §I, §I-C. [29] P. Oudeyer and F. Kaplan (2007) What is intrinsic motivation? A typology of computational approaches. Frontiers in Neurorobotics 1. External Links: ISSN 1662-5218 Cited by: §I. [30] P. Oudeyer and F. Kaplan (2008) How can we define intrinsic motivation ?. In The 8th International Conference on Epigenetic Robotics, Brighton, United Kingdom. Cited by: §I. [31] P. Oudeyer and F. Kaplan (2009) How can we define intrinsic motivation?. Cited by: §I-C. [32] S. Pateria, B. Subagdja, A. Tan, and C. Quek (2022) Hierarchical Reinforcement Learning: A Comprehensive Survey. ACM Computing Surveys 54 (5), p. 1–35. External Links: ISSN 0360-0300, 1557-7341, Document Cited by: §I. [33] Raspberry Pi (Trading) Ltd. (2021) Raspberry Pi Pico Datasheet. Cited by: §IV. [34] F. E. Ritter, F. Tehranchi, and J. D. Oury (2019) ACT-R: A cognitive architecture for modeling cognition. WIREs Cognitive Science 10 (3), p. e1488. External Links: ISSN 1939-5078, 1939-5086, Document Cited by: §I. [35] R. M. Ryan and E. L. Deci (2000) Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemporary Educational Psychology 25 (1), p. 54–67. External Links: ISSN 0361-476X, Document Cited by: §I, §I. [36] S. S. Saha, S. S. Sandha, and M. Srivastava (2022) Machine Learning for Microcontroller-Class Hardware: A Review. IEEE Sensors Journal 22 (22), p. 21362–21390. External Links: ISSN 1558-1748, Document Cited by: §I. [37] A. Saxena, A. Jain, O. Sener, A. Jami, D. K. Misra, and H. S. Koppula (2015) RoboBrain: Large-Scale Knowledge Engine for Robots. arXiv. External Links: 1412.0691, Document Cited by: §I. [38] P. Skelly Hierarchical Reinforcement Learning with Function Approximation for Adaptive Control. p. 261. Cited by: §I. [39] M. Soori, B. Arezoo, and R. Dastres (2023) Artificial intelligence, machine learning and deep learning in advanced robotics, a review. Cognitive Robotics 3, p. 54–70. External Links: ISSN 2667-2413, Document Cited by: §I. [40] K. O. Stanley and R. Miikkulainen (2002) Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation 10 (2), p. 99–127. External Links: ISSN 1063-6560, Document Cited by: §I, §I-D. [41] F. P. Such, V. Madhavan, E. Conti, J. Lehman, K. O. Stanley, and J. Clune (2018) Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv. External Links: 1712.06567, Document Cited by: §I, §I-D. [42] X. Sun and Y. Zhang (2019) A Review of Domain Knowledge Representation for Robot Task Planning. In Proceedings of the 2019 4th International Conference on Mathematics and Artificial Intelligence, Chegndu China, p. 176–183. External Links: Document, ISBN 978-1-4503-6258-0 Cited by: §I. [43] M. Svetlik, M. Leonetti, J. Sinapov, R. Shah, N. Walker, and P. Stone (2017) Automatic Curriculum Graph Generation for Reinforcement Learning Agents. Proceedings of the AAAI Conference on Artificial Intelligence 31 (1). External Links: ISSN 2374-3468, Document Cited by: §I-B. [44] F. Svoboda, D. Nunes, M. Alizadeh, R. Daries, R. Luo, A. Mathur, S. Bhattacharya, J. S. Silva, and N. D. Lane (2020) Resource Efficient Deep Reinforcement Learning for Acutely Constrained TinyML Devices. In Research Symposium on Tiny Machine Learning, Cited by: §I. [45] T. Szydlo, P. P. Jayaraman, Y. Li, G. Morgan, and R. Ranjan (2022) TinyRL: Towards Reinforcement Learning on Tiny Embedded Devices. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta GA USA, p. 4985–4988. External Links: Document, ISBN 978-1-4503-9236-5 Cited by: §I. [46] M. Tenorth and M. Beetz (2009) KNOWROB — knowledge processing for autonomous personal robots. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 4261–4266. External Links: ISSN 2153-0866, Document Cited by: §I. [47] Y. Wang, Y. Wang, S. Patel, and D. Patel (2006) A layered reference model of the brain (LRMB). IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 36 (2), p. 124–133. External Links: ISSN 1558-2442, Document Cited by: §I. [48] C. J. C. H. Watkins and P. Dayan (1992) Q-learning. Machine Learning 8 (3-4), p. 279–292. External Links: ISSN 0885-6125, 1573-0565, Document Cited by: §I. -A Intrinsic Motivation For compactness, we abbreviate the novelty, progress, and difficulty functions as n(o,t)n(o,t), p(o,t)p(o,t), and d(o,t)d(o,t), respectively. The IM m(o,t)m(o,t) of an agent to pursue a masterable skill with o∈MOo∈ M_O as the product of three factors with: n(o,t)= n(o,t)= Ninitt=t0n(o,t−1)⋅(1−β)if executed attn(o,t−1)+γ(Nlimit−n(o,t−1))otherwise casesN_init&t=t_0\\ n(o,t_-1)·(1-β)&if executed at\;t\\ n(o,t-1)+γ(N_limit-n(o,t-1))&otherwise cases where β is the decay rate, γ is the growth rate, NinitN_init is the initial and NlimitN_limit is the maximum novelty value. p(o,t)= p(o,t)= f(o,t)⋅pscalefthreshold+poffsetif f(o,t)<fthreshold1−f(o,t)⋅poffsetif f(o,t)≥fthreshold casesf(o,t)· p_scalef_threshold+p_offset&if f(o,t)<f_threshold\\ 1-f(o,t)· p_offset&if f(o,t)≥ f_threshold cases where pscalep_scale and poffsetp_offset are scaling and offset parameters for progress. d(o,t)=log2(2+∑ok∈pred(o)f(ok,t)), d(o,t)= _2 (2+ _o_k (o)f(o_k,t) ), where f(opk,t)f(o_pk,t) is the fitness of each prerequisite skill opko_pk at time t and the set of prerequisites follows with: pred(o)=oi∈O∣ (o)=\o_i∈ O oi→o∈U∨ o_i→ o∈ U\ ∃oj∈pred(o):oi→oj∈U. ∃ o_j (o):\ o_i→ o_j∈ U\. Finally, at every discrete time step t, the agent selects the next skill to explore by maximizing the motivation function: onext(Mo,t):=argmaxo∈Mom(o,t), o_next(M_o,t):= o∈ M_o \,m(o,t), (5) where MoM_o is the available skill pool. -B Experience We define the experience E(o,t)E(o,t) to indicate whether a skill o∈Moo∈ M_o has been learned at time t. E(o,t)=1,if f(o,t)≥fthreshold,0,otherwiseE(o,t)= cases1,&if f(o,t)≥ f_threshold,\\ 0,&otherwise cases The total accumulated experience of all skills at time t is defined as: Et=max(0,∑ok∈MoE(ok,t))E_t= (0,\ _o_k∈ M_oE(o_k,t) ) (6) This metric is used to reflect the overall progress of the agent. -C Intrinsic Motivation Health Metrics Maximum Neglect. Let Rk(t)R_k(t) be the number of steps since skill k was last selected. The worst-case neglect is Nmax(t)=log10(maxkRk(t)+ε),ε=10−3.N_ (t)= _10\! ( _kR_k(t)+ ), =10^-3. (7) Selection Entropy. With cumulative counts Ck(t)C_k(t), the selection probability is pk(t)=Ck(t)∑j=1KCj(t).p_k(t)= C_k(t) _j=1^KC_j(t). (8) The normalized Shannon entropy is H(t)=−1logK∑k:pk(t)>0pk(t)logpk(t).H(t)=- 1 K _k:p_k(t)>0p_k(t) p_k(t). (9) Here H(t)∈[0,1]H(t)\!∈\![0,1] measures coverage of skill sampling, while Nmax(t)N_ (t) captures long-term skill neglect.