Paper deep dive
AlphaNIST: Emergent Assembly Skills from Binary Task Rewards
Octi Zhang, Iretiayo Akinola, Rosario Scalise, Mateo Guaman Castro, Patrick Yin, Tobias Widmer, Miles Macklin, Chris Amevor, Dylan Turpin, Dirk Van Gelder, Jan Carius, Ankur Handa, Gavriel State, Yunrong Guo, Yashraj Narang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 10/12/2026, 7:12:41 AM
Summary
The paper introduces AlphaNIST, a reinforcement learning framework that trains a single policy to solve all 20 tasks on the NIST Assembly Task Board #1 using only sparse binary task rewards. The approach leverages diverse reset states, success-guided sampling, and large-scale parallel simulation to enable emergent dexterous behaviors and multi-step planning without demonstrations or dense reward shaping, achieving a 98.2% success rate.
Entities (8)
Relation Signals (7)
AlphaNIST → solves → NIST Assembly Task Board #1
confidence 99% · The first multi-task RL policy capable of solving all 20 tasks on the NIST Assembly Task Board 1.
AlphaNIST → achievessuccessrate → 98.2%
confidence 98% · reaching 98.2% overall training success without demonstrations, progress shaping, scripted primitives, or per-task tuning.
AlphaNIST → uses → Sparse Binary Task Reward
confidence 98% · AlphaNIST trains one policy across all 20 assembly tasks with standard PPO and a binary task reward for completed assembly.
AlphaNIST → optimizesusing → PPO
confidence 96% · AlphaNIST trains one policy across all 20 assembly tasks with standard PPO
AlphaNIST → employs → Success-Guided Sampling
confidence 95% · Its training distribution combines diverse cached reset states from OmniReset [10] with beta-distributed success-guided sampling (SGS) [7]
AlphaNIST → implementsin → Isaac Lab
confidence 94% · We implement AlphaNIST in Isaac Lab [11] with GPU-parallel environments.
AlphaNIST → employs → OmniReset
confidence 93% · Its training distribution combines diverse cached reset states from OmniReset [10] with beta-distributed success-guided sampling (SGS) [7]
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The NIST Assembly Task Board #1 remains a challenging benchmark for learning-based methods in robotic manipulation. Over short horizons, it demands precise, contact-rich control of small, tight-fitting parts. Over long horizons, it requires recovery and coordination across extended action sequences. Demonstrations, task decomposition, and dense progress rewards have enabled research advances by providing guidance on how to assemble. But how much of the strategy must be specified? The progression from AlphaGo to AlphaGo Zero showed that RL at scale can discover superhuman strategies without prescribing how to win. AlphaNIST asks whether a sparse binary task reward and task diversity admitted by this task-agnostic reward can similarly let a robot discover how to assemble. In simulation, we train a single policy from scratch across all 20 assembly tasks on the NIST Assembly Task Board #1, reaching 98.2% overall training success without demonstrations, progress shaping, scripted primitives, or per-task tuning. The framework also trains policies to complete ten assembly pairs in a single episode without a prescribed order, with task success rewarded only when all ten pairs are simultaneously complete. The learned policies discover dexterous and planning-like behaviors: choosing grasps for later assembly, using surrounding geometry to reorient parts, and preparing one part, assembling another, then returning to finish the first without an explicit planner. For videos, see our website: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2610.13354v1
- Canonical: https://arxiv.org/abs/2610.13354v1
Trouble viewing inline? Open PDF directly →
Full Text
39,692 characters extracted from source content.
Expand or collapse full text
AlphaNIST: Emergent Assembly Skills from Binary Task Rewards Octi Zhang1, Iretiayo Akinola1, Rosario Scalise2, Mateo Guaman Castro2, Patrick Yin1,2 Tobias Widmer1, Miles Macklin1, Chris Amevor1, Dylan Turpin1, Dirk Van Gelder1 Jan Carius1, Ankur Handa1, Gavriel State1, Yunrong Guo1, and Yashraj Narang1 Affiliation: 1NVIDIA 2University of Washington Abstract The NIST Assembly Task Board #1 remains a challenging benchmark for learning-based methods in robotic manipulation. Over short horizons, it demands precise, contact-rich control of small, tight-fitting parts. Over long horizons, it requires recovery and coordination across extended action sequences. Demonstrations, task decomposition, and dense progress rewards have enabled research advances by providing guidance on how to assemble. But how much of the strategy must be specified? The progression from AlphaGo to AlphaGo Zero showed that RL at scale can discover superhuman strategies without prescribing how to win. AlphaNIST asks whether a sparse binary task reward and task diversity admitted by this task-agnostic reward can similarly let a robot discover how to assemble. In simulation, we train a single policy from scratch across all 20 assembly tasks on the NIST Assembly Task Board #1, reaching 98.2% overall training success without demonstrations, progress shaping, scripted primitives, or per-task tuning. The framework also trains policies to complete ten assembly pairs in a single episode without a prescribed order, with task success rewarded only when all ten pairs are simultaneously complete. The learned policies discover dexterous and planning-like behaviors: choosing grasps for later assembly, using surrounding geometry to reorient parts, and preparing one part, assembling another, then returning to finish the first without an explicit planner. For videos, see our website: https://alpha-nist.github.io/ †aftertitle: Fig. 1: Emergent multi-component assembly with a single policy. Trained with a binary task reward, AlphaNIST learns to select, grasp, reorient, and assemble diverse components, while autonomously sequencing multiple assemblies within a single episode. Shown are snapshots from a ten-component assembly sequence. I Introduction Robotic assembly is a longstanding challenge, and the NIST Assembly Task Boards are a persistent benchmark [1]; to the best of our knowledge, no learning-based approaches have solved all tasks on any of the boards. Assembly Task Board #1 (ATB1) consists of 20 assembly pair tasks (a part and its fixed mating geometry on the board) that are relevant for manufacturing, including peg insertion, gear meshing, electrical connector insertion, and nut threading. Solving this task board is exceptionally challenging for three key reasons: (1) The sequence for solving the tasks might be unspecified, and not all sequences may be efficient or valid (e.g., solving task N may obstruct the robot from accessing the parts in task N + 1). (2) Each task is long-horizon, encompassing grasping, transport, and contact-rich assembly [2]. Parts may begin in a wide range of poses and require multiple reorientations before grasping. During transport, parts may need further reorientation in-hand or through extrinsic contact. During assembly, parts may slip, become misaligned, or jam during insertion or threading [2], requiring robust recovery and reexecution. (3) Solving the tasks requires precise control during complex contact interactions. The parts themselves are small (as small as 4 m in diameter), have high geometric detail (e.g., helical threads), and have tight clearances between mating surfaces (as low as zero). Tasks such as nut threading involve learning helical motion during frictional contact between multiple mating surfaces, during which a nut can tilt, cross-thread, or jam instead of advancing along the bolt [2]. Learning-free approaches typically rely on planning and strong human priors: assembly sequence planning [3]; customized fixtures, adapters, and tooling [1, 4]; engineered motion and force primitives [5]; and tuned admittance, impedance, or force-motion controllers [4, 5]. Learning-based approaches, on the other hand, can relax some of these constraints, but have shown progress only within a limited scope. Most efforts have assumed a fixed assembly sequence, heavily constrained the initial poses of parts (e.g., beginning in an already-grasped state) [6], assumed access to accurate pose estimates, learned specialist rather than multi-task policies [2], addressed only a subset of tasks on the board [6], and/or achieved only low-to-moderate success rates by industrial standards. Success-Guided Sampling (SGS) [7], for example, solves tasks one at a time, but requires a separate policy per task and is considerably less compute-efficient. We draw inspiration from the landmark AlphaGo series in reinforcement learning, which surpassed the strongest human Go players. The progression from AlphaGo to AlphaGo Zero showed that learning from a terminal task objective could encourage superhuman decision-making over long horizons without human demonstrations [8, 9]. In this paper, we ask whether a similar approach is possible for robotics: can sparse binary task rewards, combined with appropriate training strategies, enable a robot to solve sequences of long-horizon, contact-rich manipulation tasks, perhaps leveraging dexterous behaviors that no human could feasibly specify or demonstrate? To make learning from sparse task rewards more tractable, we take a seemingly counterintuitive step: train jointly across heterogeneous assembly pairs differing in geometries, tolerances, and assembly requirements (a nut and its bolt, a gear and its gearshaft, a peg and its hole) from a broad coverage of initial states, converting MDP diversity into a resource rather than a burden. Fig. 2: The complete NIST Assembly Task Board #1 in simulation. Yellow indicates the 20 movable parts in their fully-assembled states. Guided by this insight, we develop AlphaNIST, a learning framework that trains a single policy under sparse binary task rewards across all 20 tasks on the NIST Assembly Task Board 1 (Fig. 2). The policy handles an exceptional range of initial part poses, learns from point-cloud inputs without first training specialist policies, and achieves a 98.2% overall success rate. Furthermore, the same framework can train policies to complete 10 assembly pairs in sequence, with a task reward issued only when all pairs are assembled with no prescribed order. In other words, the policy exhibits planning-like behavior. Our contributions are the following: 1. A highly-effective multi-task RL framework consisting of a novel combination of the following components: A) generation of a cache of assembly-relevant reset states [10] over heterogeneous assembly pairs, B) success-guided sampling within the cache [7], C) a unified sparse binary task reward, D) multi-GPU training at scale. These components are supported by careful ablations. All tasks share one training configuration, without per-task tuning or demonstrations. 2. The first multi-task RL policy capable of solving all 20 tasks on the NIST Assembly Task Board 1. The policy takes point clouds as input; initial part poses are sampled from a wide range, including adverse conditions (e.g., upside-down); and each task can be solved with near-perfect success rates. Positive transfer is observed, where multi-task policies can solve the 20 tasks with higher success rate than individual specialists, including three tasks that the specialists struggle to solve. 3. The first assembly policy capable of solving up to 10 diverse assembly pairs in sequence, without an explicit hierarchy or prescribed task order. Thus, this work demonstrates that sequencing behavior can emerge within a single low-level assembly policy. We observe a repertoire of unique emergent behaviors including dexterous skills such as pushing and sliding (to singulate parts), in-hand pivoting, multiple sequential regrasps, extrinsic contact (to reorient parts prior to insertion), and single-gripper nut threading. The single policy also exhibits long-horizon planning-like behavior such as bringing parts into reach, moving obstructing parts aside, and leaving a task partially-finished and then returning to it later. None of these skills and behaviors were specified. We encourage readers to watch our supplementary videos 11 1 https://alpha-nist.github.io/ for additional examples of these emergent behaviors. We plan to release our code following the review and revision period. I Related Work I-A Scalable Learning for Contact-Rich Assembly Parallel physics simulation supports experience collection at scale for on-policy methods such as PPO [11, 12], with applications ranging from dexterous in-hand manipulation [13, 14, 15, 16] to industrial assembly. GPU-accelerated physics engines such as PhysX [17] and Newton [18] support large-scale parallel simulation for robot learning. For industrial assembly, Factory [2] demonstrated real-time or faster simulation of 1,000 simultaneous nut-and-bolt interactions in Isaac Gym, later incorporated into Isaac Lab. AlphaNIST uses this simulation throughput to collect experience across heterogeneous assembly pairs. OmniReset [10] addressed state coverage through diverse, manipulation-centric resets, demonstrating long-horizon on-policy RL with fixed algorithm hyperparameters and no curricula or human demonstrations. Success-Guided Sampling (SGS) [7] addressed experience allocation by concentrating simulation near the policy’s changing capability frontier, scaling beyond one million parallel environments using sparse rewards. In its reported assembly experiments, SGS trains a separate policy for each single task. AlphaNIST retains SGS’s sampling rule but trains one multi-task policy across all 20 ATB1 tasks under a common binary task reward. Our experiments show that this shared policy learns assemblies that remain unsolved in single-task training (Fig. 5). I-B Multi-Task Reinforcement Learning Multi-task RL aims to improve learning by sharing experience across tasks. However, standard joint-training baselines have achieved lower success rates than separately trained policies on multi-task manipulation benchmarks [19]. One approach to obtaining a single multi-task policy is to distill multiple separately trained specialists into a shared policy [20]. Other approaches address the difficulty of joint training directly. PopArt [21] addresses differences in reward scales with a separate value output and normalization statistics for each task, while PCGrad [19] removes conflicting components of task gradients before combining them. However, Kurin et al. [22] show that shared-network baselines with standard regularization and stabilization can match specialized multi-task optimizers. AlphaNIST takes a simple approach to multi-task learning by exploiting the shared structure of assembly: threading, gear meshing, and insertion all receive the same binary task reward when assembly error falls below a common threshold. Under this shared objective, we train one geometry-conditioned policy and one value function with standard PPO at scale, without task-specific value heads, gradient projection, or specialist pretraining. I-C Learning-Based Robot Assembly Prior learning-based assembly systems make exploration tractable by shortening the task horizon or restricting initialization to staged, pre-grasped, or local-insertion problems [23, 24]. Factory [2] trains separate pick, place, and screw subpolicies, while AutoMate [20] combines assembly-by-disassembly, imitation learning, and curriculum-based RL to obtain specialist and generalist assembly policies. Refinery [25] improves pretrained assembly specialists through Bayesian-optimization-guided sampling of initial conditions during fine-tuning and Gaussian-mixture-model-based initialization at deployment. The refined policies can then be chained to complete multi-part assemblies. For longer sequences, ARCH [26] uses a high-level imitation-learned policy to select from a library of RL and model-based primitives. These methods structure learning through initialization, demonstrations, or prescribed subskills. OmniReset [10] uses a generic dense objective with fixed weights, including gripper–object and object–goal distance terms. Zhang et al. [7] present the first end-to-end RL solution to individual tasks from the NIST ATB1, but retain task-specific auxiliary grasp- and gripper-related shaping. Such terms can aid exploration and discourage reward exploitation, but their relative influence can change with object size, geometry, and assembly motion. Even with fixed coefficients, a new task family requires checking that the shaped reward remains aligned with successful assembly. AlphaGo Zero provides a broader motivation: terminal win/loss and search-generated policy targets support strategy discovery without human game data or intermediate reward shaping [9]. AlphaNIST asks whether binary task success can similarly support strategy discovery in continuous, contact-rich control, with one shared policy learning heterogeneous assemblies end to end. I Problem Formulation I-A Task Definition and MDP We formulate all 20 ATB1 assembly tasks [1] as a multi-task reinforcement-learning problem with one shared policy. Let e∈ℰe denote an assembly task, with |ℰ|=20|E|=20. Each task corresponds to one assembly pair: a movable part, its mating geometry, and a fully-assembled configuration. We model the problem as a finite-horizon MDP, ℳ=(,,p,r,γ,T,ρ),M=(S,A,p,r,γ,T,ρ), (1) where p(st+1∣st,at)p(s_t+1 s_t,a_t) is the transition kernel and ρ is the joint distribution over task identities and initial states. At the beginning of each episode, (e,s0)∼ρ(e,s_0) ρ. A shared policy πθ(at∣ot) _θ(a_t o_t) acts on partial observations oto_t of the simulator state sts_t and maximizes J(θ)=ρ,p,πθ[∑t=0T−1γtrt].J(θ)=E_ρ,p, _θ [ _t=0^T-1γ^tr_t ]. (2) Task variation is represented through surface point clouds in oto_t, rather than an explicit discrete task identifier. IV Method IV-A AlphaNIST Training Framework Point cloud Proprioception + last action pose 77 ∣ twist 66 ∣ joints 99 ∣ action 88 two-layer ELU encoder3×npts3× n_pts256256128128RN150150150150∥\|278278linear256256LNlinear10241024SWlinear256256++identityLNlinear10241024SWlinear256256++identityLNlinear256256LNlinear10241024SWlinear256256++identityLNlinear10241024SWlinear256256++identityLNlinear88Gaussian action at∈ℝ8a_t ^8arm Δq1:7 q_1:7gripperlinear11value Vϕ(ot)∈ℝV_φ(o_t) Fig. 3: Policy architecture. Actor and critic have the same architecture with separate parameters. Inputs are drawn once; each network contains two consecutive residual blocks. RN: running normalization; LN: layer normalization; SW: Swish activation; nptsn_pts: number of input points. AlphaNIST trains one policy across all 20 assembly tasks with standard PPO and a binary task reward for completed assembly. Its training distribution combines diverse cached reset states from OmniReset [10] with beta-distributed success-guided sampling (SGS) [7], while a common binary task reward defines successful assembly across all tasks. The three components play distinct roles: Reset bank:Which states are available?Beta sampling:Which states are sampled?Binary task reward:What counts as success? array[]r@ lReset bank:&Which states are available?\\ Beta sampling:&Which states are sampled?\\ Binary task reward:&What counts as success? array IV-B Reset Coverage and Experience Allocation We adapt OmniReset [10] to heterogeneous assembly by caching the assembly identity together with each reset state: reset=(ei,s¯i)i=1M.D_reset=\(e_i, s_i)\_i=1^M. (3) Here, M is the total number of entries across all 20 assemblies. The assembly identity eie_i selects the part and mating-geometry assets and their inertial properties, while s¯i s_i specifies the robot, board, and movable-part state. Sampling an entry therefore selects both the assembly and its initial configuration. The cached bank spans ungrasped workspace, pickup, transport, near-mating, and partially assembled configurations (Fig. 4), enabling GPU-parallel initialization. These reset families define regions of the training distribution, not a prescribed sequence; the policy receives no stage identifier. Fig. 4: Diverse assembly reset states. Examples of resets across tasks, ranging from random object poses to in-hand grasps and partial assemblies. Let p^i p_i be the empirical success rate over the most recent L episodes initialized from entry (ei,s¯i)(e_i, s_i). Under beta sampling, entry i is selected with probability qiq_i: qi∝εq+p^i(1−p^i),i=1,…,M,q_i _q+ p_i(1- p_i), i=1,…,M, (4) normalized over the entire bank. Sampling i∼Cat()i (q) selects an assembly–reset pair with ρ(ei,s¯i)=qiρ(e_i, s_i)=q_i. The kernel favors intermediate success rates, while εq>0 _q>0 keeps every entry sampleable. Each episode’s outcome updates p^i p_i for its sampled entry. The uniform ablation sets qi=1/Mq_i=1/M; the bank, policy, reward, and optimizer remain unchanged. Algorithm 1 summarizes training. IV-C Binary Full-Assembly Objective Across geometries, the assembly objective is the same: complete the assembly. For each instance, an assembly-relative coordinate Φe(α) _e(α), α∈[0,1]α∈[0,1], parameterizes the part in the mating frame, with Φe(1) _e(1) denoting its fully assembled pose. Let dasme(s)d_asm^e(s) be the distance between the part’s current relative pose and Φe(1) _e(1). For a single-pair episode, the binary task reward is rtsucc=1,dasme(st+1)<ϵ,0,otherwise.r_t^succ= cases1,&d_asm^e(s_t+1)<ε,\\ 0,&otherwise. cases (5) where ϵ=0.001mε=0.001\,m. The success reward is issued once, and the episode terminates. Intermediate progress receives no task reward. We supplement the binary task reward with a control cost to discourage large action magnitudes, rapid joint motion, and excessive torque. This regularization biases learning toward smoother, lower-effort behaviors that are more compatible with safe deployment on physical robots, without providing information about assembly progress. rt=rtsucc⏟binary task reward−c(∥at∥22+∥q˙t∥22+∥τt∥22)⏟control cost,c≥0.r_t= r_t^succ_binary task reward- c ( a_t _2^2+ q_t _2^2+ _t _2^2 )_control cost, c≥ 0. (6) Here, ata_t, q˙t q_t, and τt _t are the action, robot joint velocity, and applied joint torque. The control penalties are evaluated at every step, with a single coefficient c fixed across tasks. They do not measure assembly progress; the binary task reward specifies successful completion. Episodes end on assembly success, at horizon T, or when an arm joint exceeds its hardware velocity or torque limit. In multi-pair episodes, the success reward remains zero until every active pair is assembled, with no prescribed pair order or intermediate completion reward. Fig. 5: Training across heterogeneous assembly pairs substantially improves learning from a sparse binary task reward. At 64K environments across 25k iterations: (a) compares 20-task success rates for one shared policy versus 20 separately trained policies. The shared policy (first row) achieved above 98% success rate across all tasks, including three precision assemblies that corresponding specialists rarely solve. Dense shaping does not exhibit the same benefit. Panel (b) shows overall success, note that the separate-policy success rate is average across all 20 per-task policies; Asset notation: N, G, R, and P denote nuts, gears, rods, and pegs, respectively; numeric suffixes indicate nominal diameters in millimeters, while s, m, and l indicate gear size. "Sparse reward" denotes Eq. (6); "dense reward" additionally includes the shaping terms in Eq. (7). Fig. 6: Joint sparse training converges faster than separate policies. Curves compare one shared policy trained on all 20 tasks with the mean of 20 policies trained separately, one per task. Fig. 7: Sparse task rewards benefit strongly from parallel experience. Mean success ± standard deviation over 5 evaluation seeds for 25k-iteration checkpoints, with 32 trials per task per seed across 20 assemblies. Color denotes reward; solid and dashed lines denote beta and uniform training-reset sampling. (a) Board-assisted reorientation(waterproof) (b) Release-and-regrasp for alignment(d-sub) (c) Center-to-tip grasp transfer(peg 8m) (d) Side-caging for gear placement(medium gear) (e) Edge-to-flat nut reorientation(nut m12) (f) One-sided tapping for threading(nut m4) (g) Shoveling to a better pickup position(rod 8m) (h) Full-circle nut threading(nut m16) Fig. 8: One shared policy discovers diverse contact strategies from a sparse binary task reward. Each panel proceeds from left to right. (a) Waterproof connector: board contact helps reorient the part before insertion. (b) D-sub connector: the part is set down and regrasped for alignment with its socket. (c) 8 m peg: the part is dropped from a center grasp and regrasped near one end, leaving its remaining length free for insertion. (d) Medium gear: the part pivots from a side grasp into a horizontal pose over its mounting post. (e) M12 nut: the nut is rolled from its edge onto a flat face before placement on the bolt. (f) M4 nut: repeated one-sided taps advance the nut along the thread. (g) 8 m rod: the fingertip shovels the rod to a position where it is easier to lift. (h) M16 nut: the gripper moves around the nut, using fingertip contact to turn it along the bolt. (a) Bring USB-A closer, assemble the 8 m rod, then return (b) Making small turns threading M12 to avoid RJ45 collision Fig. 9: Emergent coordination in multi-pair assembly. (a) The policy brings the USB-A connector from a distant corner to a nearby working position, leaves it there while assembling the 8 m rod, then returns to insert the connector. (b) During M12 nut threading, the end effector makes small turns to avoid touching the neighboring RJ45 connector. Each sequence proceeds from left to right. IV-D Shared Policy and Robot Control We implement AlphaNIST in Isaac Lab [11] with GPU-parallel environments. Each PPO batch combines rollouts from assembly–reset pairs sampled using Eq. (4). The policy, reward, controller, and training configuration are shared across all 20 tasks. The policy observes the robot state and surface point clouds of the movable part, mating geometry, and robot, without a discrete task identifier. We use separate actor and critic SimBa networks [27, 28], summarized in Fig. 3. Each combines a point-cloud embedding with the proprioception and last action shown in the figure. Algorithm 1 AlphaNIST Heterogeneous Assembly Learning 1: reset=(ei,s¯i)i=1MD_reset=\(e_i, s_i)\_i=1^M; history length L; batch size N; shared πθ,Vϕ _θ,V_φ 2: initialize each rolling success rate p^i←0 p_i← 0 3: for each PPO iteration do 4: ℬ←∅B← 5: while |ℬ|<N|B|<N do 6: for all completed episodes n do 7: yn←[full assembly]y_n 1[full assembly] for entry ini_n 8: p^in← p_i_n← mean of its latest L outcomes 9: end for 10: update reset probabilities q using Eq. (4) 11: for all environments n requiring reset do 12: in∼Cat()i_n (q); (en,sn,0)←(ein,s¯in)(e_n,s_n,0)←(e_i_n, s_i_n) 13: end for 14: an,t∼πθ(⋅∣on,t),∀na_n,t _θ(· o_n,t), 9.24994pt∀ n 15: step all environments and append transitions to ℬB 16: end while 17: update πθ _θ and VϕV_φ using standard PPO on ℬB 18: end for We simulate a 7-DoF Franka Emika Panda with identified inertial and drivetrain parameters and a parallel-jaw gripper with one driven and one mechanically coupled passive finger. At 25Hz25\,Hz, the policy outputs a 7D relative joint position target (qttarget=qt+Δqtq_t^target=q_t+ q_t) and a 1D gripper command. An implicit impedance controller uses the manufacturer’s libfranka stiffness and damping profile with hardware-matched armature and joint friction. Joint-speed limits are 2.175rad/s2.175\,rad/s for joints 1–4 and 2.61rad/s2.61\,rad/s for joints 5–7; torque limits are 87Nm87\,N\,m and 12Nm12\,N\,m, respectively. These constraints keep actions within a hardware-realistic envelope, preventing the policy from exploiting unrealistic speeds or torques. Similar hardware-aligned simulation has enabled transfer of assembly policies to the same Franka Panda platform [29, 20]. IV-E Distributed Training and Scale We scale synchronous PPO across 1–256K environments, with 4K per GPU in multi-GPU runs. Each GPU samples its local reset bank using rolling success estimates. Actor and critic gradients are averaged across GPUs before each optimizer step. Each iteration collects 32 steps per environment and optimizes for 5 epochs with 4 minibatches each. Network architecture and PPO settings are fixed across scales. V Experiments We organize the experiments around three questions: (Q1) Heterogeneity: Under a binary task reward, how does joint training compare with training each task separately? (Q2) Scaling: How do parallel experience and reset-state sampling affect learning from a binary task reward? (Q3) Reward formulation: How does a binary task reward compare with dense assembly-progress shaping as heterogeneous training scales? We additionally explore multi-pair assembly and inspect the learned policy for behaviors that arise without direct supervision. For heterogeneity, we compare one policy trained across all 20 ATB1 tasks with one separately trained policy per task, using matched per-run interaction budgets and otherwise identical settings. We use 64K environments per run to keep the comparison affordable; the shared policy reaches 98.2% training success at this scale. For parallel scaling, GPUs, environments, and rollout batch size increase together while the task distribution, reward, policy, and environment remain fixed. The sampling ablation compares against uniform sampling over the same reset bank instead of beta sampling. For the scaling evaluation, all checkpoints use stochastic actions and a common frozen mixed-reset bank, sampled uniformly within each task. Success requires full assembly within 15 seconds without a simultaneous safety failure. The reward ablation adds two dense assembly-progress terms to Eq. (6): rtdense=rt+c2rtreach+c3rtalign,c2,c3≥0,r_t^dense=r_t+c_2r_t^reach+c_3r_t^align, c_2,c_3≥ 0, (7) where rtreachr_t^reach rewards end-effector proximity to the part centroid and rtalignr_t^align rewards alignment with the target assembly orientation. These are common shaping terms in manipulation and assembly RL [2, 20, 10]. We set c2=c3=0.01c_2=c_3=0.01, matching the coefficients used in OmniReset [10]. Across all experiments, success rates cover all reset configurations. TABLE I: Success rate for multi-pair assembly Each policy is trained and evaluated on the same number of active assembly pairs, from 1–10. AlphaNIST solves these tasks without a prescribed order or explicit task decomposition. For example, when there are 10 active pairs, the policy receives a success reward only when all 10 are in their assigned board slots. # Active Pairs 1 2 3 4 5 6 7 8 9 10 Success Rate (%) 99 97 95 93 92 90 89 87 86 84 V-A Assembly-Pair Heterogeneity Enables Learning From randomly initialized states, separately-trained policies have ≤ 0.3% success on M4 nut threading, as well as 4 m peg insertion and rod insertion. The main obstacle is that the small parts are difficult to grasp reliably from the table. In contrast, the shared policy produces successful grasp-and-assembly rollouts on these same tasks without task-specific changes. This result suggests that shared experience can help with assemblies that are difficult to learn separately. This benefit depends strongly on the reward formulation. At their final checkpoints, the 20 separately trained dense-reward policies reach a mean success rate of 78.3%, ranging from 45.0% on M4 threading to 93.6% on the medium gear. Joint dense training reaches 17.8% overall success at 25K iterations, with low completion across tasks (Fig. 5). One possible explanation is that dense shaping terms favor different partial solutions across geometries, whereas the binary task reward gives a unified full-assembly objective for every task. Joint training reaches higher success at lower total cost. At 64K parallel environments across 25K iterations, the shared policy trained with the sparse binary task reward reaches 98.2% training success in 410 GPU-hours (Fig. 6), while the 20 separately trained policies average 81.4% and consume 16.1K total GPU-hours, 39.3× more device-hours. V-B Sparse Binary Task Rewards Benefit from Scale Performance with the sparse binary task reward improves rapidly with increased parallel experience (Fig. 7). With beta sampling, evaluation success rises from 38.6% at 1K environments to 78.9% at 4K and 86.9% at 8K. With dense reward shaping, success remains at 11.8–12.0% over 1K–4K environments. Beta sampling provides an additional gain over uniform reset sampling. At 256K environments, success with the sparse binary task reward reaches 97.4% with beta sampling versus 84.3% with uniform sampling; the corresponding results with dense reward shaping are 77.4% and 34.3%. While the advantage of the sparse task objective is already visible at modest scale, beta sampling provides further gains as parallel experience increases. V-C Emergent Strategies over Short Horizons The policy exhibits a broad range of dexterous strategies across assemblies (Fig. 8), including grasp adaptation, extrinsic reorientation, and contact-driven assembly. It releases and reacquires parts when the initial grasp is unsuitable for insertion, uses surrounding geometry to change object orientation, and discovers intermittent tapping and pivoting motions. These behaviors emerge within a single policy, without demonstrations or rewards specifying the individual maneuvers. For a peg, a successful pickup may not produce a grasp suitable for insertion; the policy drops a center-held peg and regrasps it near one end, exposing enough shaft length for complete insertion. For a nut, repeated taps on one side engage and advance the thread, using intermittent contact to make assembly progress. V-D Emergent Strategies over Long Horizons Beyond the main single-assembly experiments, we train policies with multiple assembly pairs per episode. The binary task reward is issued only when every active pair is assembled; episodes allow 15 seconds per pair and terminate upon success. Table I shows the success rates for episodes with up to ten active pairs. This multi-pair setting reveals coordination over substantially longer horizons (Fig. 9). The policy need not finish assembling a pair before turning to another, and its approach to one assembly changes based on neighboring parts. These planning-like behaviors—taking actions whose utility is realized later and adapting one assembly to the state of another—emerge without an explicit planner, policy switching, or prescribed sequence. VI Limitations First, all results are in simulation. Hardware-matched velocity and torque limits constrain the learned motions, but do not establish robustness to perception error, latency, or contact-model mismatch. Transfer and repeatability will be evaluated on the physical board using sensor-derived observations and randomized contact and latency models. Second, while applying the 10-pair policy to successive groups of parts could offer a route to completing the full board, extending our multi-pair framework to 20 parts remains an exciting research goal. Achieving this at scale, under a single terminal task reward, will likely benefit from more efficient simulation and learning methods. VII Conclusion AlphaNIST shows that heterogeneous contact-rich assembly can be learned with a single policy using a binary task objective, diverse reset coverage, and large-scale reinforcement learning. In simulation, one policy learns all 20 ATB1 tasks without demonstrations, prescribed stages, or per-task tuning, while outperforming separately trained specialists on the most difficult small-part assemblies. Across all tested scales, the binary task objective outperforms the evaluated dense reward shaping. The learned policy discovers grasp adaptation, extrinsic reorientation, tapping, threading, and longer-horizon coordination without direct supervision. In multi-pair assembly, it can prepare one part, complete another, and return to finish the first, without an intermediate assembly reward or a planner prescribing the sequence. More broadly, these findings challenge the assumption that task complexity and diversity must be reduced to make robot learning tractable. Instead, they show that diverse experience under a shared objective can itself become a source of reusable dexterity and coordination. We view this work as a critical step toward general-purpose assembly policies that discover how to solve complex physical tasks, rather than requiring those strategies to be specified in advance. References [1] K. Kimble et al. (2020) Benchmarking protocols for evaluating small parts robotic assembly systems. IEEE Robot. Automat. Lett. 5 (2), p. 883–889. External Links: Document Cited by: §I, §I, §I-A. [2] Y. Narang et al. (2022) Factory: fast contact for robotic assembly. In Proc. Robot.: Sci. Syst., External Links: Document Cited by: §I, §I, §I-A, §I-C, §V. [3] T. Lozano-Pérez and R. H. Wilson (1993) Assembly sequencing for arbitrary motions. In Proc. IEEE Int. Conf. Robot. Automat., p. 527–532. Cited by: §I. [4] W. Lian, T. Kelch, D. Holz, A. Norton, and S. Schaal (2021) Benchmarking off-the-shelf solutions to robotic assembly tasks. In Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., p. 791–798. External Links: Document Cited by: §I. [5] A. S. Morgan, Q. Bateux, M. Hao, and A. M. Dollar (2023) Towards generalized robot assembly through compliance-enabled contact formations. In Proc. IEEE Int. Conf. Robot. Automat., p. 8010–8016. External Links: Document Cited by: §I. [6] J. Luo et al. (2021) Robust multi-modal policies for industrial assembly via reinforcement learning and demonstrations: a large-scale study. In Proc. Robot.: Sci. Syst., External Links: Document Cited by: §I. [7] O. Zhang, M. Guaman Castro, P. Yin, A. Gupta, R. Scalise, and B. Boots (2026) A balanced data diet: addressing the bottleneck in mega-scale RL for robot control. External Links: Link Cited by: item 1, §I, §I-A, §I-C, §IV-A. [8] D. Silver et al. (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529, p. 484–489. External Links: Document Cited by: §I. [9] D. Silver et al. (2017) Mastering the game of Go without human knowledge. Nature 550, p. 354–359. External Links: Document Cited by: §I, §I-C. [10] P. Yin et al. (2026) Emergent dexterity via diverse resets and large-scale reinforcement learning. In Proc. Int. Conf. Learn. Representations, External Links: Link Cited by: item 1, §I-A, §I-C, §IV-A, §IV-B, §V. [11] M. Mittal et al. (2025) Isaac Lab: a GPU-accelerated simulation framework for multi-modal robot learning. Note: arXiv:2511.04831 Cited by: §I-A, §IV-D. [12] K. Zakka, Q. Liao, B. Yi, L. L. Lay, K. Sreenath, and P. Abbeel (2026) mjlab: a lightweight framework for GPU-accelerated robot learning. Note: arXiv:2601.22074 Cited by: §I-A. [13] M. Andrychowicz et al. (2020) Learning dexterous in-hand manipulation. Int. J. Robot. Res. 39 (1), p. 3–20. External Links: Document Cited by: §I-A. [14] H. Qi et al. (2023) General in-hand object rotation with vision and touch. In Proc. Conf. Robot Learn., p. 2549–2564. Cited by: §I-A. [15] R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk (2024) DextrAH-RGB: visuomotor policies to grasp anything with dexterous hands. Note: arXiv:2412.01791 Cited by: §I-A. [16] A. Handa et al. (2023) Dextreme: transfer of agile in-hand manipulation from simulation to reality. In Proc. IEEE Int. Conf. Robot. Automat. (ICRA), p. 5977–5984. Cited by: §I-A. [17] NVIDIA GPU simulation. Note: PhysX SDK 5.4.1 documentationAccessed: Sep. 14, 2026 External Links: Link Cited by: §I-A. [18] The Newton Contributors (2025) Newton: GPU-accelerated physics simulation for robotics and simulation research. External Links: Link Cited by: §I-A. [19] T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn (2020) Gradient surgery for multi-task learning. In Proc. Adv. Neural Inf. Process. Syst., p. 5824–5836. Cited by: §I-B. [20] B. Tang et al. (2024) AutoMate: specialist and generalist assembly policies over diverse geometries. In Proc. Robot.: Sci. Syst., External Links: Document Cited by: §I-B, §I-C, §IV-D, §V. [21] M. Hessel, H. Soyer, L. Espeholt, W. Czarnecki, S. Schmitt, and H. van Hasselt (2019) Multi-task deep reinforcement learning with PopArt. In Proc. AAAI Conf. Artif. Intell., Vol. 33, p. 3796–3803. External Links: Document Cited by: §I-B. [22] V. Kurin, A. De Palma, I. Kostrikov, S. Whiteson, and P. K. Mudigonda (2022) In defense of the unitary scalarization for deep multi-task learning. In Proc. Adv. Neural Inf. Process. Syst., p. 12169–12183. External Links: Document Cited by: §I-B. [23] G. Schoettler et al. (2020) Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards. In Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., p. 5548–5555. External Links: Document Cited by: §I-C. [24] C. Florensa, D. Held, M. Wulfmeier, M. Zhang, and P. Abbeel (2017) Reverse curriculum generation for reinforcement learning. In Proc. Conf. Robot Learn., p. 482–495. Cited by: §I-C. [25] B. Tang et al. (2025) Refinery: active fine-tuning and deployment-time optimization for contact-rich policies. Note: arXiv:2510.11019 Cited by: §I-C. [26] J. Sun et al. (2025) ARCH: hierarchical hybrid learning for long-horizon contact-rich robotic assembly. In Proc. Conf. Robot Learn., External Links: Link Cited by: §I-C. [27] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., p. 770–778. External Links: Document Cited by: §IV-D. [28] H. Lee et al. (2025) SimBa: simplicity bias for scaling up parameters in deep reinforcement learning. In Proc. Int. Conf. Learn. Representations, External Links: Link Cited by: §IV-D. [29] B. Tang et al. (2023) IndustReal: transferring contact-rich assembly tasks from simulation to reality. In Proc. Robot.: Sci. Syst., External Links: Document Cited by: §IV-D.