Paper deep dive
FoldBack: Self-Correcting Masked Generative Policy for Long-Horizon Garment Folding
Lipeng Zhuang, Shiyu Fan, Yingdong Ru, Zhuo He, Florent P. Audonnet, Paul Henderson, Gerardo Aragon Camarasa
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 10/13/2026, 3:18:34 AM
Summary
The paper introduces FoldBack, a self-correcting masked generative policy for long-horizon garment folding. It addresses the issue of missed or slipped grasps by implementing three inference-time decisions: Key-Action-Aligned Inference (KAAI) for timing refinement, Verified Rollback to restore pre-grasp configurations upon failure, and Structured Plan Repair to regenerate failed trajectory segments while avoiding previous failure locations. FoldBack achieves 75.2% final folding success and 0.837 final-mask IoU on 33 real garments, outperforming the strongest prior baseline.
Entities (10)
Relation Signals (8)
FoldBack → includescomponent → Verified Rollback
confidence 95% · FoldBack initiates the verified rollback procedure described below.
FoldBack → includescomponent → Key-Action-Aligned Inference
confidence 95% · FoldBack determines when to refine and verify the trajectory (Key-Action-Aligned Inference, Sec. III-A)
FoldBack → includescomponent → Structured Plan Repair
confidence 95% · FoldBack repairs the trajectory in two steps... Structured Plan Repair
FoldBack → outperforms → DP3
confidence 95% · FoldBack achieves 75.2% final folding success... versus 45.7%... for the strongest prior baseline.
FoldBack → trainedon → Flat'n'Fold
confidence 90% · We train the MGP backbone on Flat’n’Fold [3]
FoldBack → uses → MGP-Long
confidence 90% · FoldBack adopts the Masked Generative Policy (MGP) [9] as its generative backbone.
Verified Rollback → uses → PointNet++
confidence 85% · Thus, a PointNet-based [38] binary classifier fψ predicts pk,r
FoldBack → uses → VQ-VAE
confidence 85% · with a VQ-VAE tokenizer [37] trained to map between continuous action trajectories and this discrete representation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We present FoldBack, a self-correcting masked generative policy for long-horizon garment folding. Existing long-trajectory policies may continue after a missed or slipped grasp even when the garment has not reached the intended configuration. We structure FoldBack's recovery mechanisms around three inference-time decisions: when to refine and verify, how to roll back, and where and how to retry. FoldBack aligns refinement and grasp verification with pick-and-place events, returns the robot to a retryable pre-grasp configuration while preserving successful grasps, and selectively regenerates the failed segment and selected future actions while avoiding previous failed grasp locations. To our knowledge, FoldBack is the first editable full-trajectory policy to unify these decisions, enabling failed interactions to be detected, undone, and repaired before execution continues, without recovery demonstrations or base-policy retraining. Across 33 real garments from six categories, FoldBack achieves 75.2% final folding success and 0.837 final-mask IoU, versus 45.7% and 0.689 for the strongest prior baseline.
Tags
Links
- Source: https://arxiv.org/abs/2610.10462v1
- Canonical: https://arxiv.org/abs/2610.10462v1
Trouble viewing inline? Open PDF directly →
Full Text
46,049 characters extracted from source content.
Expand or collapse full text
FoldBack: Self-Correcting Masked Generative Policy for Long-Horizon Garment Folding Thanks: The authors are with the University of Glasgow, Glasgow, United Kingdom. 2658047z@student.gla.ac.uk Lipeng Zhuang Shiyu Fan Yingdong Ru Zhuo He Affiliation: Florent P. Audonnet, Paul Henderson, Gerardo Aragon Camarasa Abstract We present FoldBack, a self-correcting masked generative policy for long-horizon garment folding. Existing long-trajectory policies may continue after a missed or slipped grasp even when the garment has not reached the intended configuration. We structure FoldBack’s recovery mechanisms around three inference-time decisions: when to refine and verify, how to roll back, and where and how to retry. FoldBack aligns refinement and grasp verification with pick-and-place events, returns the robot to a retryable pre-grasp configuration while preserving successful grasps, and selectively regenerates the failed segment and selected future actions while avoiding previous failed grasp locations. To our knowledge, FoldBack is the first editable full-trajectory policy to unify these decisions, enabling failed interactions to be detected, undone, and repaired before execution continues, without recovery demonstrations or base-policy retraining. Across 33 real garments from six categories, FoldBack achieves 75.2%75.2\% final folding success and 0.8370.837 final-mask IoU, versus 45.7%45.7\% and 0.6890.689 for the strongest prior baseline. Fig. 1: Failure handling in garment folding. Unlike baseline policies that may continue or stall after a failed grasp, FoldBack verifies the grasp outcome, rolls back, repairs the trajectory, and retries before advancing. I Introduction Garment folding is a long-horizon manipulation task that requires executing a sequence of precise single-arm or bimanual pick-and-place actions. Each action changes the garment configuration, which in turn determines the conditions for subsequent folding actions. A missed or slipped grasp therefore invalidates subsequent folding actions [1]. For example, a robot may close its gripper and lift without successfully moving the garment, leaving it in a configuration incompatible with the next folding action. Reliable folding not only requires long-horizon action generation but also a mechanism for detecting grasp failures and recovering from them before execution advances. Visuomotor imitation learning enables robots to learn complex behaviors from demonstrations [2, 1, 3]. Short-horizon policies generate action chunks myopically given recent observations, enabling reactive execution [4, 5, 6, 7]. However, grasp failures can produce out-of-distribution observations that cause subsequent predictions to stall, drift, or lose coherence with the overall folding strategy. Long-horizon generative policies generate complete, coherent action sequences and can refine future action tokens as observations arrive [8, 9]; yet executed action tokens remain fixed in the trajectory history without verifying the intended manipulation outcome. Thus, execution may advance to the next folding stage without revisiting the failed interaction. Runtime monitors can detect such failures but do not themselves revise the policy trajectory [10, 11, 12], while recovery-aware methods often rely on additional recovery data, policy adaptation, or large-model reasoning [1, 13, 14, 15, 16]. Motivated by these limitations, we introduce FoldBack, an outcome-gated masked generative policy for long-horizon garment folding that supports recovery from grasp failures. FoldBack represents the full action trajectory as discrete tokens and maintains a grasp-verified history, a tentative grasp-critical window, and an editable future. Executing a grasp-critical window does not immediately add it to history. The window is added only after all required grasps are verified; otherwise, the history remains unchanged and the failed window stays editable (Fig. 1). FoldBack is then organized around three inference-time decisions which constitute the contributions of this paper: 1. when to refine and verify (Sec. I-A), 2. how to roll back after a failure (Sec. I-B), and 3. where and how to retry (Sec. I-C). This enables recovery without recovery demonstrations or base-policy retraining. Indeed, across 33 garments from 6 categories, FoldBack achieves 75.2%75.2\% final folding success and a mean final-mask IoU of 0.8370.837 across single-arm and bimanual folding tasks, outperforming the strongest prior baseline by 29.529.5 percentage points in final folding success and 0.1480.148 in mean final-mask IoU. FoldBack also generalizes to unseen garment instances, initial orientations, and non-canonical configurations as shown in Sec. IV. I Related Work Learning-based garment manipulation. Garment-folding policies typically compose prescribed primitives or generate local action segments [17, 18, 19]. SpeedFolding [20] and UniFolding [2] execute parameterized primitives, while BiFold [21] predicts a language-conditioned pick-and-place action at each step. Model-based methods learn cloth dynamics for planning [22, 23], while UniClothDiff [24] uses diffusion models for generative state estimation and dynamics prediction within model-predictive control. Other generative policies model distributions over action primitives (DeformPAM [25]), closed-loop or local manipulation trajectories (FoldNet [1] and HALO [26]), or stage-wise point-cloud trajectories (MetaFold [27]). These methods are effective for local or stage-wise manipulation, but long-horizon folding requires consistent execution across multiple strategies with both single- and bimanual actions. FoldBack addresses this with full-trajectory masked generation for an editable plan (Sec. I), Key-Action-Aligned Inference for event-aligned refinement and grasp verification (Sec. I-A), Verified Rollback to keep failed windows revisable (Sec. I-B), and Structured Plan Repair to selectively revise the affected trajectory (Sec. I-C). Imitation learning. Imitation learning learns visuomotor policies from demonstrations. Short-horizon policies generate local action sequences and replan from observations, including ACT [5], Diffusion Policy [6], DP3 [7], 3D Diffuser Actor [28], BeT [29], and VQ-BeT [30]. Recent methods improve temporal consistency or inference efficiency through sequence-level generation and action-chunk refinement [8, 31, 32, 33, 34], but do not maintain an editable task plan. MGP-Long [9] instead generates a complete trajectory, fixes executed tokens as history, and refines uncertain future tokens. Thus, it cannot revisit failed grasps. FoldBack addresses this by adding grasp-critical windows to the grasp-verified history only after verification (Sec. I-B) and regenerating failed windows during plan repair (Sec. I-C). Failure monitoring and recovery. Runtime monitoring methods detect anomalous behavior or insufficient progress but do not generate recovery actions [10, 11, 12]. Some recovery-aware methods instead learn corrective behavior from additional data. For instance, FoldNet uses KG-DAgger recovery trajectories [1], while RACER [13] trains a VLM-supervised, language-conditioned policy on generated recovery trajectories and fine-grained annotations. More general correction frameworks include CycleVLA [15], which combines VLA-based progress prediction with VLM-guided backtracking, while FAR [16] adapts a diffusion policy from test-time failures and perturbs retry actions. Bellman-Guided Retrials [35] performs progress-guided retrials, while Rewind-IL [36] rewinds policies to verified checkpoints. As described in Sec. I-B and Sec. I-C, FoldBack instead reopens failed windows within an editable full trajectory and selectively repairs the affected trajectory while preserving grasp-verified history. It requires neither failure-recovery demonstrations, base-policy adaptation, nor auxiliary LLM/VLM reasoning. Fig. 2: Overview of FoldBack. Key-Action-Aligned Inference aligns trajectory refinement with pick-and-place events; grasp monitoring and verified rollback determine whether a grasp-critical window is added to the verified trajectory history or kept editable for recovery; structured plan repair then remasks and resamples the editable trajectory after failure. I Method We formulate garment folding as long-horizon action generation conditioned on an observation t=(t,t)o_t=(P_t,s_t) at time t, where tP_t is an observed point cloud and ts_t is the robot’s proprioceptive state. An action t∈ℝ16a_t ^16 consists of, for each arm r∈L,Rr∈\L,R\, a 3-D end-effector position, a 4-D orientation quaternion, and a binary gripper state gtr∈0,1g_t^r∈\0,1\. From human demonstrations, we learn a policy that predicts a T-step action trajectory (t,…,t+T−1)(a_t,…,a_t+T-1) conditioned on the current observation to_t. FoldBack adopts the Masked Generative Policy (MGP) [9] as its generative backbone. MGP represents a T-step action trajectory using N discrete tokens =(y1,…,yN)y=(y_1,…,y_N), with a VQ-VAE tokenizer [37] trained to map between continuous action trajectories and this discrete representation, and N<TN<T. Given a partially masked token sequence and observation oto_t, the Masked Generative Transformer (MGT) predicts masked tokens in parallel. MGP initially generates the full trajectory from a fully masked sequence, executes decoded actions, and subsequently remasks selected future tokens using updated observations. In the original MGP inference procedure, executed action tokens become fixed trajectory history regardless of whether their intended physical outcomes were achieved. FoldBack instead keeps a grasp-critical window tentative until its grasp outcome is verified. If all required grasps succeed, the window is added to the grasp-verified trajectory history; otherwise, the history remains unchanged and the failed window remains editable for recovery. As shown in Fig. 2, FoldBack determines when to refine and verify the trajectory (Key-Action-Aligned Inference, Sec. I-A), whether to add the executed window to history or return the robot to the pre-grasp configuration (Sec. I-B), and how to repair the editable trajectory after failure (Sec. I-C). We train the MGP backbone on Flat’n’Fold [3] following its original two-stage procedure [9]. The VQ-VAE tokenizer is first trained with position and orientation reconstruction losses, binary cross-entropy on gripper states, and the VQ commitment loss, after which it is frozen. The MGT is then trained to recover action tokens that are randomly masked or replaced with alternative codebook indices, conditioned on the corresponding observation. FoldBack requires no additional policy training or recovery demonstrations. I-A Key-Action-Aligned Inference (KAAI) KAAI ensures trajectory refinement and grasp verification happen after pick-and-place transitions of garment folding rather than at uniform execution intervals. Since pick and place events mark the main physical transitions in folding, after which the garment configuration can change substantially, this effectively enables the robot to perform visual servoing at key moments in the trajectory. Following initial trajectory generation, FoldBack extracts these key actions from the decoded gripper-state sequences (g1r,…,gTr)(g_1^r,…,g_T^r). An open-to-closed transition of the gripper defines a Pick, whereas a closed-to-open transition defines a Place. This yields J key actions at times t1<⋯<tJt_1<·s<t_J. Each key action k defines an action window k=tk−4,…,tk+3W_k=\t_k-4,…,t_k+3\. Let ℐk⊆1,…,NI_k \1,…,N\ denote the smallest set of token indices whose decoded actions cover kW_k. In bimanual manipulation, the two arm-specific Pick transitions are treated as a single key action, whose action window spans from the first gripper-closing event to the end of the corresponding joint lift. A window whose key action is a Pick is grasp-critical because it establishes a grasp that the rest of the folding stage depends on. Before and after executing each key-action window kW_k, FoldBack refines the remaining unexecuted tokens using the latest observation. After each refinement, FoldBack re-decodes the gripper-state sequence and recomputes the transition time tjt_j, action window jW_j, and token set ℐjI_j for each unexecuted key action j. The Pick/Place order and arm assignment remain fixed, while their timing may shift as the trajectory is refined. We use two MGP mask-refinement iterations at each update. I-B Grasp-outcome monitoring and Verified Rollback FoldBack verifies each required grasp after lifting, and then adds the corresponding grasp-critical window to the trajectory history only when all grasps succeed. In sequential pick-and-place manipulation, each grasp is verified after its own lift. In bimanual manipulation, the first arm maintains its grasp while the second performs its pick; both arms then lift, and both grasps are verified. If any grasp fails, FoldBack initiates the verified rollback procedure described below. Fig. 3: Repair under an asymmetric bimanual grasp failure. Local Grasp-Outcome Monitor At verification point k, FoldBack evaluates each arm r whose grasp must be verified. Using the gripper poses, we extract pre-pick and post-lift local point clouds, k,rpreP_k,r^pre and k,rliftP_k,r^lift, from a fixed region below the fingertips before grasp closure and after lifting, respectively. The two point clouds are expressed in their respective gripper frames. This representation focuses the monitor on changes in local garment geometry, largely excludes the gripper, and requires no garment segmentation. Thus, a PointNet-based [38] binary classifier fψf_ψ predicts pk,r=fψ(k,rpre,k,rlift)∈[0,1],p_k,r=f_ψ\! (P_k,r^pre,P_k,r^lift )∈[0,1], (1) where ψ denotes the monitor parameters and pk,rp_k,r is the predicted grasp-success probability. The grasp is verified as successful when pk,r≥τmonp_k,r≥ _mon, with τmon=0.50 _mon=0.50 selected on the validation set and fixed for all evaluations. We train fψf_ψ using binary cross-entropy on successful Flat’n’Fold demonstrations. A positive pair contains k,rpreP_k,r^pre and its corresponding k,rliftP_k,r^lift. A negative pair replaces k,rliftP_k,r^lift with a local point cloud from a nearby pre-pick frame at the location where the post-lift point cloud would be taken. Because the garment remains on the table in that frame, the resulting pair approximates a missed grasp without requiring physical failure trajectories. Training pairs are constructed independently for each arm, and one classifier is shared across single-arm and bimanual stages. The trained policy and monitor remain fixed during deployment. Grasp-Verified Trajectory History FoldBack does not immediately add the executed tokens of a grasp-critical window to the trajectory history. Instead, it keeps the token subsequence ℐky_I_k tentative until post-lift verification is complete, while the unexecuted future trajectory remains editable. Let vk∈0,1v_k∈\0,1\ denote the grasp-outcome monitor decision for kW_k, where vk=1v_k=1 indicates that all associated grasps are verified as successful and vk=0v_k=0 otherwise. If vk=1v_k=1, FoldBack appends ℐky_I_k to the grasp-verified trajectory history ℋverH^ver; otherwise, ℋverH^ver remains unchanged and the failed window remains editable for plan repair; neither the failed attempt nor the rollback motion is added to the history. Tokens executed outside the current grasp-critical window remain fixed as trajectory history. For a bimanual manipulation window, the joint token subsequence ℐky_I_k is saved to the trajectory history only if both arms’ grasps are verified as successful. Verified rollback FoldBack records the executed end-effector motion from the pre-grasp to the post-lift configuration. When vk=0v_k=0, each failed gripper opens and the recorded motion is executed in reverse, restoring the pre-grasp configuration. In a bimanual manipulation, both arms return in coordination to their pre-grasp configurations; if only one grasp fails, the successful gripper remains closed to preserve its grasp. At the restored configuration, FoldBack captures an updated point cloud and robot state. The procedure is policy-agnostic and requires no base-policy retraining. After rollback, action-chunking policies replan from the updated observation, whereas masked long-trajectory policies can additionally retain grasp-verified history and regenerate affected editable tokens. I-C Structured Trajectory Plan Repair Verified rollback restores a retryable configuration but does not revise the trajectory. This is because the unexecuted actions were generated under the assumption that the failed grasp would succeed. When a grasp-critical window kW_k fails verification, FoldBack repairs the trajectory in two steps. Distance-guided remasking The unexecuted token positions after ℐkI_k are grouped into segments 1,…,J−kS_1,…,S_J-k, one for each subsequent key-action window. 1S_1 contains the actions immediately after the failed window up to the next key-action window; each subsequent segment spans between consecutive key-action windows. The final segment additionally includes any remaining actions after the last key-action window. FoldBack fully remasks ℐkI_k and assigns each segment ℓS_ the masking ratio, defined as the fraction of action tokens masked and regenerated: ρℓ=ρbase+(ρnear−ρbase)αℓ−1, _ = _base+ ( _near- _base )α -1, (2) where ρbase _base is the masking ratio used by MGP-Long during trajectory refinement, ρnear>ρbase _near> _base is the ratio applied to 1S_1, and α∈(0,1)α∈(0,1) controls the rate of decay. The masking ratio therefore decreases toward ρbase _base as the segment distance from the failure increases. Within each segment ℓS_ , FoldBack remasks the ρℓ _ -fraction of tokens with the lowest stored confidence γilast _i^last, defined as the probability assigned to the current token yiy_i when it was most recently generated. We denote the resulting set of remasked positions, including ℐkI_k, by ℳM. FoldBack regenerates the tokens at these positions using the observation captured after the robot returns to the pre-grasp configuration, while the grasp-verified prefix and unselected future tokens remain fixed as anchors. During subsequent mask-refinement iterations, the stored confidence of each regenerated token is updated with its newly predicted probability. We use ρnear=0.80 _near=0.80, ρbase=0.30 _base=0.30, and α=0.50α=0.50 in all experiments. Failure-aware candidate resampling A regenerated trajectory plan may repeat the grasp that just failed. FoldBack therefore draws eight repair candidates for ℳM from the post-rollback observation. Each arm maintains an episode-level memory ℱrF_r of its failed table-plane pick locations, to which the current failure is added before sampling. A candidate is rejected if its proposed pick for an arm that just failed lies within an exclusion radius ϵ=0.04mε=0.04\,m of any location in ℱrF_r. Under an asymmetric bimanual failure, only candidates for the failed arm are rejected based on proximity to previous failed pick locations, while the successful arm retains its grasp and is not resampled. FoldBack samples up to three candidate batches to obtain a valid retry trajectory. If no valid candidate is found, the recovery episode is marked as unsuccessful and the manipulation terminates at the restored pre-grasp configuration. Let γi(q) _i^(q) denote the probability assigned to candidate (q∈1,…,8q∈\1,…,8\) at regenerated position i∈ℳi . Each candidate is scored by its mean confidence over the regenerated positions: C(q)=1|ℳ|∑i∈ℳγi(q).C^(q)= 1|M| _i _i^(q). (3) For single-arm failures or bimanual failures of both grasps, FoldBack executes the valid candidate with the highest C(q)C^(q). Fig. 4: Real-world FoldBack recovery. (a) Short-sleeved T-shirt: recovery from repeated grasp failures during multi-stage folding. (b) Long-sleeved T-shirt: recovery from an asymmetric bimanual failure while preserving the successful grasp. For an asymmetric bimanual failure (Fig. 3), FoldBack constructs a time-aligned window for each valid resampled motion. Each candidate combines the stored motion of the successful arm, whose gripper remains closed, with the regenerated motion of the failed arm. Candidates that fail inverse kinematics or violate joint limits are discarded. Each feasible candidate is re-encoded by the frozen tokenizer and evaluated in one batched MGT pass, with ℐkI_k masked and all other candidate tokens fixed. Let γ¯i(q) γ_i^(q) denote the probability assigned to the re-encoded bimanual token at i∈ℐki _k. The candidate score is C¯(q)=1|ℳ|(∑i∈ℳ∖ℐkγi(q)+∑i∈ℐkγ¯i(q)), C^(q)= 1|M| ( _i _k _i^(q)+ _i _k γ_i^(q) ), (4) which replaces the original confidence terms for the failed window with those of the executable bimanual candidate. FoldBack executes the feasible candidate with the highest C¯(q) C^(q). If no feasible candidate remains, it samples another candidate batch, subject to the same three-batch limit. Once all required grasps are verified after the retry, the re-encoded bimanual subsequence at ℐkI_k is appended to ℋverH^ver, the corresponding stored confidences are updated, and the remaining trajectory is refined using the new post-lift observation. TABLE I: Real-world long-horizon garment-folding performance. ‘VR’, ‘R&R’, and ‘SR’ denote Verified Rollback, Rollback-and-Repair, and Structured Repair, respectively. Method Napkin Towel Pants SC RS FS IoU SC RS FS IoU SC RS FS IoU DP3 71.7 – 56.7 0.717 60.5 – 53.3 0.703 66.7 – 60.0 0.706 SKIL 75.0 – 63.3 0.755 65.8 – 60.0 0.743 60.0 – 53.3 0.731 MGP-Long 78.3 – 70.0 0.786 73.7 – 66.7 0.791 73.3 – 66.7 0.764 FoldBack w/o R&R 83.3 – 73.3 0.828 81.6 – 73.3 0.815 80.0 – 66.7 0.786 DP3 + VR 85.0 53.8 76.7 0.823 78.9 60.0 73.3 0.814 76.7 55.6 66.7 0.788 FoldBack w/o SR 90.0 70.0 83.3 0.853 86.8 66.7 80.0 0.846 83.3 60.0 73.3 0.826 FoldBack(ours) 93.3 90.9 90.0 0.872 94.7 85.7 93.3 0.876 86.7 83.3 80.0 0.851 Method Short-sleeved T-Shirt Short-sleeved Shirt Long-sleeved T-Shirt Average SC RS FS IoU SC RS FS IoU SC RS FS IoU SC RS FS IoU DP3 34.8 – 6.7 0.502 30.4 – 6.7 0.469 26.7 – 6.7 0.458 44.3 – 35.2 0.610 SKIL 42.0 – 13.3 0.558 33.3 – 13.3 0.501 34.7 – 6.7 0.548 48.7 – 39.0 0.656 MGP-Long 50.7 – 20.0 0.605 43.5 – 13.3 0.551 40.0 – 13.3 0.540 56.3 – 45.7 0.689 FoldBack w/o R&R 63.8 – 33.3 0.692 56.5 – 26.7 0.648 54.7 – 26.7 0.652 67.2 – 53.3 0.750 DP3 + VR 65.2 42.1 26.7 0.674 55.1 47.1 20.0 0.627 52.0 45.0 20.0 0.619 66.3 48.9 51.4 0.738 FoldBack w/o SR 75.4 61.9 46.7 0.761 66.7 55.0 33.3 0.716 66.7 63.6 40.0 0.711 76.2 61.9 62.9 0.795 FoldBack(ours) 82.6 79.2 60.0 0.811 76.8 78.3 53.3 0.782 77.3 77.8 60.0 0.793 83.9 80.6 75.2 0.837 IV Experiments We evaluate FoldBack in the real world, aiming to answer four questions: (1) Can it improve the execution of long-horizon garment-folding strategies? (Sec. IV-A), (2) Can verified rollback and trajectory plan repair recover from controlled one- and two-corner grasp slips? (Sec. IV-B), (3) How well does the policy generalize to unseen garments and initial configurations? (Sec. IV-C), (4) How do FoldBack’s inference-time design choices affect execution and recovery? (Sec. IV-D). All policies are trained on Flat’n’Fold robot demonstrations. Real-world setup We follow the Flat’n’Fold setup which includes a Baxter dual-arm robot and a single ZED2 RGB-D camera. Each point cloud is cropped to the workspace and downsampled to 4,096 points. From the point cloud and robot proprioception, the policy predicts a 16-D action vector as defined in Sec. I. Unless stated otherwise, each trial starts with a flattened garment in the workspace. We evaluate on 33 garments from six categories, including napkins of two sizes, three towel instances, pants, short-sleeved T-shirts, long-sleeved T-shirts, and short-sleeved shirts. The garment instances vary in size. The evaluated multi-stage strategies may alternate between single-arm and bimanual manipulation. Some categories use multiple strategies; for towels, the initial orientation determines whether the policy performs a two- or three-stage fold. We train one policy per category jointly on all demonstrated strategies for that category. Model variants We evaluate FoldBack w/o Rollback-and-Repair, which retains KAAI but disables grasp-outcome verification, rollback, and trajectory plan repair, and FoldBack w/o Structured Repair, which retains KAAI, grasp-verified history, and rollback but replaces structured repair with MGP-Long’s [9] confidence-based remasking of the failed window and remaining future, followed by a single repair candidate. Baselines We compare against the action-chunking policies 3D Diffusion Policy (DP3) [7] and SKIL [4], and the full-trajectory generative policy MGP-Long [9]. We also evaluate DP3 + Verified Rollback, which equips DP3 with the same grasp-outcome monitor and verified rollback as FoldBack without modifying or retraining the base policy. During normal execution, these baselines perform inference every 12 executed actions, whereas FoldBack refines its trajectory at key-action boundaries. Upon a detected failure, the DP3 variant returns to the pre-grasp configuration, discards the unexecuted actions in the current action chunk, and generates a new action chunk from the resulting observation. DP3, SKIL, and MGP-Long do not explicitly detect grasp failures or initiate retries, and therefore continue execution after a failed grasp. Evaluation metrics A recovery episode starts when a grasp failure is first detected in a folding stage and allows up to three rollback-and-retry attempts; further failures during those retries belong to the same episode. The episode ends when the stage completes, and the trial terminates if the stage is still incomplete after the third retry. Recovery success rate (RS) is the fraction of episodes whose stage completes within this budget, and first-retry success rate (FRS) is the fraction of episodes that complete on the first retry. Stage completion rate (SC) is the fraction of required folding stages completed across all trials. We take a reference mask from the final top-down frame of a successful demonstration of the same garment and folding strategy, and we compute the IoU in image coordinates. Final-mask IoU is averaged over all trials, including incomplete ones. A trial counts as a final folding success (FS) if every stage completes and the final IoU is at least 0.800.80. IV-A Long-Horizon Garment Folding We conduct 30 trials on napkins (15 trials per napkin size) and 15 trials for each other category (i.e. pants, t-shirts, etc.). Table I shows that MGP-Long is the strongest baseline without recovery. KAAI alone improves average final success from 45.7%45.7\% to 53.3%53.3\%, while full FoldBack reaches 75.2%75.2\% final success and 0.8370.837 final-mask IoU, achieving the best final success in all six categories; representative recovery executions are shown in Fig. 4. Structured repair further improves final success from 62.9%62.9\% to 75.2%75.2\%. Moreover, the gains are larger on the evaluated five-stage strategies. Against MGP-Long, the stage-completion gains are 13.413.4–15.015.0 points on the evaluated two-stage folds and 31.931.9–37.337.3 points on the three five-stage garment categories. TABLE I: Recovery from controlled grasp-slip perturbations on towel folding. ‘FB’, ‘VR’, ‘SR’, and ‘F’ denote FoldBack, verified rollback, structured repair, and fixed failed window. One-Corner Slip Two-Corner Slip Method Det.↑ Pres. ↑ RS ↑ FS ↑ Det.↑ RS ↑ FS ↑ DP3 N/A N/A N/A 6.7 N/A N/A 0.0 DP3 + VR 86.7 38.5 53.8 33.3 86.7 46.2 20.0 MGP-Long N/A N/A N/A 13.3 N/A N/A 6.7 FB w/o Rollback 93.3 92.9 57.1 40.0 93.3 42.9 13.3 FB w/o SR + F 93.3 64.3 50.0 26.7 100.0 40.0 20.0 FB w/o SR 93.3 71.4 64.3 53.3 100.0 53.3 33.3 FoldBack (ours) 93.3 100.0 85.7 73.3 100.0 80.0 66.7 TABLE I: Generalization performance, reported as final folding success rate (%) / final-mask IoU. S-T, L-T, and S-S denote short-sleeve T-shirt, long-sleeve T-shirt, and short-sleeve shirt, respectively. Initial Orientation (Pants) Unseen Instance Non-Canonical Method Seen Mod. Large Nap. Tow. Pants S-T L-T S-S Avg. Nap. Tow. DP3 60.0/0.706 46.7/0.661 33.3/0.603 60.0/0.704 40.0/0.642 50.0/0.671 10.0/0.486 0.0/0.421 10.0/0.458 28.3/0.564 40.0/0.648 33.3/0.612 MGP-Long 66.7/0.764 60.0/0.724 40.0/0.642 70.0/0.774 50.0/0.710 60.0/0.731 20.0/0.596 10.0/0.521 10.0/0.530 36.7/0.644 53.3/0.704 46.7/0.674 FoldBack (ours) 80.0/0.851 73.3/0.817 60.0/0.752 90.0/0.863 80.0/0.833 70.0/0.812 60.0/0.780 50.0/0.756 50.0/0.742 66.7/0.798 80.0/0.826 73.3/0.798 IV-B Recovery under Controlled Grasp-Slip Perturbations Natural failures vary across trials, so we inject controlled one- or two-corner slips at a predetermined pick event during bimanual towel folding. During lifting, an operator dislodges either one corner (producing one failed and one successful grasp) or both corners (causing both grasps to fail). Each trial contains one perturbation and no subsequent intervention. We conduct 15 trials for each evaluated method–condition pair. Failure-detection rate (Det.) is the fraction of injected perturbations detected by the monitor, while recovery success rate is the fraction of detected recovery episodes whose affected stage is completed within three recovery attempts. For one-corner slips, successful-grasp preservation rate (Pres.) is the fraction of detected asymmetric failures in which the successful grasp remains held from failure detection until recovery or termination. To isolate the effect of grasp-verified history, we compare FoldBack without Structured Repair with FoldBack without Structured Repair with a Fixed Failed Window, which treats the failed window as executed history and keeps it fixed after rollback, refining only the remaining future trajectory. We also evaluate FoldBack without Rollback, which uses the same grasp-outcome monitoring, grasp-verified history, and plan repair but repairs the trajectory directly from the failure configuration without returning to the pre-grasp state. We further evaluate the frozen monitor on held-out garments, using its first prediction after each lift to classify the grasp outcome. Labels are manually annotated independently of the monitor output, and the threshold is selected on the validation set and fixed for testing. Across 96 failed and 352 successful grasps, the monitor achieves 92.7%92.7\% failed-grasp recall and an arm-wise false-positive rate of 4.5%4.5\%. This indicates that the monitor detects most grasp failures while rarely triggering unnecessary recovery after a successful grasp. Verified rollback improves DP3’s final folding success under both perturbations (Table I). Under one- and two-corner slips, keeping the failed window editable rather than fixing it after execution raises recovery success by 14.314.3 and 13.313.3 percentage points, respectively, while physical rollback yields corresponding gains of 28.628.6 and 37.137.1 points over FoldBack without Rollback. Full FoldBack achieves recovery success of 85.7%85.7\% and 80.0%80.0\% and improves final folding success over FoldBack without Structured Repair by 20.020.0 and 33.333.3 points, respectively. Full FoldBack preserves the successful grasp in 100%100\% of detected asymmetric failures because it keeps the successful gripper closed and reuses that arm’s recorded motion while regenerating only the failed arm. FoldBack without Structured Repair regenerates both arms jointly and preserves the successful grasp in only 71.4%71.4\% of detected asymmetric failures. IV-C Generalization Table I compares DP3, MGP-Long, and FoldBack on unseen initial orientations, held-out garment instances, and non-canonical initial configurations. Qualitative examples are provided in the accompanying video. Initial-orientation generalization. Using pants, we evaluate at the training orientation, where the garment is placed diagonally on the table as in the demonstrations, and two unseen rotation ranges relative to this orientation: moderate (5∘−20∘5 \!-\!20 ) and large (20∘−30∘20 \!-\!30 ). Trials at the training orientation reuse the pants trials from Table I, while each unseen rotation range contains 15 trials per method. FoldBack achieves final success rates of 80.0%80.0\%, 73.3%73.3\%, and 60.0%60.0\%, compared with 66.7%66.7\%, 60.0%60.0\%, and 40.0%40.0\% for MGP-Long. Unseen garment instances. For each category, we hold out one physical garment from training and retrain each method on the remaining demonstrations. Ten trials per held-out instance yield 60 trials per method. FoldBack achieves 66.7%66.7\% average final success, compared with 36.7%36.7\% for MGP-Long and 28.3%28.3\% for DP3, and performs best in all six categories. Non-canonical initial configurations. We initialize napkins and towels in near-flat configurations with local wrinkles or a partially folded corner, requiring the robot to flatten before folding. This evaluates whether the learned flatten-and-fold behavior transfers to unseen non-canonical initial configurations. Across 15 trials per category, FoldBack achieves 80.0%80.0\% and 73.3%73.3\% final success on napkins and towels, respectively, versus 53.3%53.3\% and 46.7%46.7\% for MGP-Long. Examples are provided in the accompanying video. TABLE IV: Three ablations on towel folding. (a) Schedule (b) Remasking (c) Resampling Variant SC ↑ FS ↑ Variant RS ↑ FS ↑ Variant FRS ↑ RS ↑ FS ↑ Fixed-12 73.7 66.7 Full future 75.0 80.0 Single 55.6 77.8 80.0 Uniform 76.3 66.7 Failed window 71.4 80.0 Conf.-ranked 66.7 83.3 86.7 KAAI 81.6 73.3 Dist.-guided 85.7 93.3 Failure-aware 71.4 85.7 93.3 IV-D Ablation Studies We conduct three ablations on towel-folding with 15 trials per variant (Table IV). To isolate event alignment from update frequency, we compare three refinement strategies using the same trained policy with grasp monitoring and rollback-and-repair disabled. Fixed-12 refines every 12 executed actions, Uniform distributes the same number of refinements as KAAI uniformly over the horizon, and KAAI refines before and after each key-action window. KAAI achieves the highest stage completion and final success; because Uniform uses the same number of refinements, this comparison isolates the benefit of event alignment. We next compare three remasking strategies. Full future regenerates the entire unexecuted trajectory, whereas Failed window regenerates only the failed key-action window. Distance-guided fully remasks the failed window and progressively lowers the masking ratio over subsequent segments, prioritizing low-confidence tokens. It achieves the highest recovery success (85.7%85.7\%) and final folding success (93.3%93.3\%). Finally, we compare three candidate-selection strategies. Single draws one repair candidate, Confidence ranking selects the highest-confidence candidate among Q=8Q=8, and Failure-aware additionally rejects candidates whose failed-arm pick lies within the exclusion region around previous failed grasps. Failure-aware achieves the highest first-retry, recovery, and final success rates: 71.4%71.4\%, 85.7%85.7\%, and 93.3%93.3\%. Batched resampling takes 58ms58\,ms per recovery event versus 37ms37\,ms for a single candidate, adding little inference latency. V Conclusion We introduced FoldBack, a self-correcting masked generative framework for long-horizon garment folding. FoldBack maintains a grasp-verified trajectory history so that failed grasp windows can be updated rather than becoming fixed trajectory history. Upon failure, it returns the robot to a retryable pre-grasp configuration, selectively repairs the trajectory with distance-guided remasking, and resamples candidates while avoiding previously failed grasp locations. Across six garment categories, FoldBack improves stage completion, recovery success, and final folding performance over action-chunking and full-trajectory baselines. Future work will extend FoldBack to more challenging initial configurations that require a separate flattening phase before folding can begin. References [1] Y. Chen, B. Xiao, and H. Wang (2026) FoldNet: learning generalizable closed-loop policy for garment folding via keypoint-driven asset and demonstration synthesis. IEEE Robotics Autom. Lett. 11 (3), p. 3820–3827. Cited by: §I, §I, §I, §I. [2] H. Xue, Y. Li, W. Xu, H. Li, D. Zheng, and C. Lu (2023) UniFolding: towards sample-efficient, scalable, and generalizable robotic garment folding. In Conference on Robot Learning (CoRL), Vol. 229, p. 3321–3341. Cited by: §I, §I. [3] L. Zhuang, S. Fan, Y. Ru, F. P. Audonnet, P. Henderson, and G. Aragon-Camarasa (2025) Flat’n’fold: A diverse multi-modal dataset for garment perception and manipulation. In IEEE International Conference on Robotics and Automation (ICRA), p. 7937–7944. Cited by: §I, §I. [4] S. Wang, J. You, Y. Hu, J. Li, and Y. Gao (2025) SKIL: semantic keypoint imitation learning for generalizable data-efficient manipulation. CoRR. Cited by: §I, §IV. [5] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems (RSS), Cited by: §I, §I. [6] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. Int. J. Robotics Res. 44 (10-11), p. 1684–1704. Cited by: §I, §I. [7] Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu (2024) 3D diffusion policy: generalizable visuomotor policy learning via simple 3d representations. In Robotics: Science and Systems (RSS), Cited by: §I, §I, §IV. [8] Z. Xian, N. Gkanatsios, T. Gervet, T. Ke, and K. Fragkiadaki (2023) ChainedDiffuser: unifying trajectory diffusion and keypose prediction for robotic manipulation. In Conference on Robot Learning (CoRL), Vol. 229, p. 2323–2339. Cited by: §I, §I. [9] L. Zhuang, S. Fan, F. P. Audonnet, Y. Ru, G. Aragon-Camarasa, and P. Henderson (2025) Masked generative policy for robotic control. CoRR abs/2512.09101. Cited by: §I, §I, §I, §I, §IV, §IV. [10] C. Agia, R. Sinha, J. Yang, Z. Cao, R. Antonova, M. Pavone, and J. Bohg (2024) Unpacking failure modes of generative policies: runtime monitoring of consistency and progress. arXiv preprint arXiv:2410.04640. Cited by: §I, §I. [11] C. Xu, T. K. Nguyen, E. Dixon, C. Rodriguez, P. Miller, R. Lee, P. Shah, R. Ambrus, H. Nishimura, and M. Itkina (2025) Can we detect failures without failure data? uncertainty-aware runtime failure detection for imitation learning policies. arXiv preprint arXiv:2503.08558. Cited by: §I, §I. [12] C. Gokmen, D. Ho, and M. Khansari (2023) Asking for help: failure prediction in behavioral cloning through value approximation. In IEEE International Conference on Robotics and Automation, ICRA, p. 5821–5828. Cited by: §I, §I. [13] Y. Dai, J. Lee, N. Fazeli, and J. Chai (2025) Racer: rich language-guided failure recovery policies for imitation learning. In IEEE International Conference on Robotics and Automation (ICRA), p. 15657–15664. Cited by: §I, §I. [14] Z. Liu, A. Bahety, and S. Song (2023) Reflect: summarizing robot experiences for failure explanation and correction. arXiv preprint arXiv:2306.15724. Cited by: §I. [15] C. Ma, K. Lu, G. Yang, J. Liu, S. Xu, B. Byrne, I. Havoutis, N. Trigoni, and A. Markham (2026) Cyclevla: proactive self-correcting vision-language-action models via subtask backtracking and minimum bayes risk decoding. arXiv preprint arXiv:2601.02295. Cited by: §I, §I. [16] H. Hao, S. N. Syed, J. Ichnowski, and J. Schneider (2026) FAR: failure-aware retry for test-time recovery and continual policy improvement. arXiv preprint arXiv:2607.01111. Cited by: §I, §I. [17] K. Mo, C. Xia, X. Wang, Y. Deng, X. Gao, and B. Liang (2023) Foldsformer: learning sequential multi-step cloth manipulation with space-time attention. IEEE Robotics Autom. Lett. 8 (2), p. 760–767. Cited by: §I. [18] A. Canberk, C. Chi, H. Ha, B. Burchfiel, E. Cousineau, S. Feng, and S. Song (2023) Cloth funnels: canonicalized-alignment for multi-purpose garment manipulation. In IEEE International Conference on Robotics and Automation (ICRA), p. 5872–5879. Cited by: §I. [19] T. Weng, S. M. Bajracharya, Y. Wang, K. Agrawal, and D. Held (2021) FabricFlowNet: bimanual cloth manipulation with a flow-based policy. In Conference on Robot Learning (CoRL), Vol. 164, p. 192–202. Cited by: §I. [20] Y. Avigal, L. Berscheid, T. Asfour, T. Kröger, and K. Goldberg (2022) SpeedFolding: learning efficient bimanual folding of garments. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 1–8. Cited by: §I. [21] O. Barbany, A. Colomé, and C. Torras (2025) BiFold: bimanual cloth folding with language guidance. In IEEE International Conference on Robotics and Automation (ICRA), p. 5652–5659. Cited by: §I. [22] X. Lin, Y. Wang, Z. Huang, and D. Held (2021) Learning visible connectivity dynamics for cloth smoothing. In Conference on Robot Learning (CoRL), Vol. 164, p. 256–266. Cited by: §I. [23] Z. Huang, X. Lin, and D. Held (2022) Mesh-based dynamics with occlusion reasoning for cloth manipulation. In Robotics: Science and Systems (RSS), Cited by: §I. [24] T. Tian, H. Li, B. Ai, X. Yuan, Z. Huang, and H. Su (2025) Diffusion dynamics models with generative state estimation for cloth manipulation. CoRR abs/2503.11999. Cited by: §I. [25] W. Chen, H. Xue, F. Zhou, Y. Fang, and C. Lu (2025) Deformpam: data-efficient learning for long-horizon deformable object manipulation via preference-based action alignment. In IEEE International Conference on Robotics and Automation (ICRA), p. 6896–6903. Cited by: §I. [26] Y. Wang, R. Wu, Y. Chen, J. Wang, J. Liang, Z. Zhu, H. Geng, J. Malik, P. Abbeel, and H. Dong (2025) DexGarmentLab: dexterous garment manipulation environment with generalizable policy. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 38. Cited by: §I. [27] H. Chen, J. Li, R. Wu, Y. Liu, Y. Hou, Z. Xu, J. Guo, C. Gao, Z. Wei, S. Xu, J. Huang, and L. Shao (2025) MetaFold: language-guided multi-category garment folding framework via trajectory generation and foundation model. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 4339–4346. Cited by: §I. [28] T. Ke, N. Gkanatsios, and K. Fragkiadaki (2024) 3D diffuser actor: policy diffusion with 3D scene representations. Note: arXiv preprint arXiv:2402.10885 Cited by: §I. [29] N. M. M. Shafiullah, Z. J. Cui, A. Altanzaya, and L. Pinto (2022) Behavior transformers: cloning k modes with one stone. Note: arXiv preprint arXiv:2206.11251 Cited by: §I. [30] S. Lee, Y. Wang, H. Etukuru, H. J. Kim, N. M. M. Shafiullah, and L. Pinto (2024) Behavior generation with latent actions. Note: arXiv preprint arXiv:2403.03181 Cited by: §I. [31] A. Prasad, K. Lin, J. Wu, L. Zhou, and J. Bohg (2024) Consistency policy: accelerated visuomotor policies via consistency distillation. In Robotics: Science and Systems, Cited by: §I. [32] Q. Zhang, Z. Liu, H. Fan, G. Liu, B. Zeng, and S. Liu (2025) Flowpolicy: enabling fast and robust 3d flow-based policy via consistency flow matching for robot manipulation. In AAAI Conference on Artificial Intelligence (AAAI), Vol. 39, p. 14754–14762. Cited by: §I. [33] K. Black, A. Z. Ren, M. Equi, and S. Levine (2025) Training-time action conditioning for efficient real-time chunking. arXiv preprint arXiv:2512.05964. Cited by: §I. [34] K. Black, M. Y. Galliker, and S. Levine (2025) Real-time execution of action chunking flow policies. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §I. [35] M. Du, A. Khazatsky, T. Gerstenberg, and C. Finn (2024) To err is robotic: rapid value-based trial-and-error during deployment. CoRR abs/2406.15917. Cited by: §I. [36] G. Zheng, S. Seenivasan, M. Johnson-Roberson, and W. Zhi (2026) Rewind-il: online failure detection and state respawning for imitation learning. CoRR abs/2604.16683. Cited by: §I. [37] A. Van Den Oord O. Vinyals et al. (2017) Neural discrete representation learning. Advances in neural information processing systems 30. Cited by: §I. [38] C. R. Qi, H. Su, K. Mo, and L. J. Guibas (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 652–660. Cited by: §I-B.