Paper deep dive

KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition

Gaoge Han, Zhengqing Gao, Ziwen Li, Jiaxin Huang, Shaoli Huang, Fakhri Karray, Mingming Gong, Tongliang Liu

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 68

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/22/2026, 5:54:31 AM

Summary

KineVLA is a vision-language-action (VLA) framework designed for kinematics-aware robotic manipulation. It addresses the limitation of existing VLA models that struggle with fine-grained kinematic specifications (e.g., orientation, trajectory) by introducing a bi-level action representation that decouples goal-level invariance from kinematics-level variability. The model utilizes bi-level reasoning tokens and mutual information regularization to align language instructions with hierarchical action tokens, demonstrating superior performance on kinematics-sensitive benchmarks in both simulation and real-world settings.

Entities (5)

KineVLA · framework · 100%Bi-Level RVQ-VAE · model-architecture · 95%LIBERO · datasetplatform · 95%OpenVLA · baseline-model · 95%Realman-75 · robotic-platform · 95%

Relation Signals (3)

KineVLA → builton → OpenVLA

confidence 95% · We build KineVLA on top of OpenVLA (Kim et al., 2025) as the base vision–language.

KineVLA → evaluatedon → LIBERO

confidence 95% · Extensive experiments on LIBERO and a Realman-75 robot demonstrate that KineVLA consistently outperforms strong VLA baselines

KineVLA → utilizes → Bi-Level RVQ-VAE

confidence 95% · we present our method, comprising a bi-level RVQ-VAE and KineVLA.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:In this paper, we introduce a novel kinematics-rich vision-language-action (VLA) task, in which language commands densely encode diverse kinematic attributes (such as direction, trajectory, orientation, and relative displacement) from initiation through completion, at key moments, unlike existing action instructions that capture kinematics only coarsely or partially, thereby supporting fine-grained and personalized manipulation. In this setting, where task goals remain invariant while execution trajectories must adapt to instruction-level kinematic specifications. To address this challenge, we propose KineVLA, a vision-language-action framework that explicitly decouples goal-level invariance from kinematics-level variability through a bi-level action representation and bi-level reasoning tokens to serve as explicit, supervised intermediate variables that align language and action. To support this task, we construct the kinematics-aware VLA datasets spanning both simulation and real-world robotic platforms, featuring instruction-level kinematic variations and bi-level annotations. Extensive experiments on LIBERO and a Realman-75 robot demonstrate that KineVLA consistently outperforms strong VLA baselines on kinematics-sensitive benchmarks, achieving more precise, controllable, and generalizable manipulation behaviors.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

67,923 characters extracted from source content.

Expand or collapse full text

KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Gaoge Han, Zhengqing Gao, Ziwen Li, Jiaxin Huang, Shaoli Huang, Fakhri Karray, Mingming Gong, Tongliang Liu 1 Abstract In this paper, we introduce a novel kinematics- rich vision-language-action (VLA) task, in which language commands densely encode diverse kine- matic attributes (such as direction, trajectory, ori- entation, and relative displacement) from initia- tion through completion, at key moments, unlike existing action instructions that capture kinemat- ics only coarsely or partially, thereby support- ing fine-grained and personalized manipulation. In this setting, where task goals remain invari- ant while execution trajectories must adapt to instruction-level kinematic specifications. To ad- dress this challenge, we propose KineVLA, a vision-language-action framework that explicitly decouples goal-level invariance from kinematics- level variability through a bi-level action repre- sentation and bi-level reasoning tokens to serve as explicit, supervised intermediate variables that align language and action. To support this task, we construct the kinematics-aware VLA datasets spanning both simulation and real-world robotic platforms, featuring instruction-level kine- matic variations and bi-level annotations. Exten- sive experiments on LIBERO and a Realman-75 robot demonstrate that KineVLA consistently out- performs strong VLA baselines on kinematics- sensitive benchmarks, achieving more precise, controllable, and generalizable manipulation be- haviors. The code and dataset will be released publicly. 1. Introduction Vision-Language-Action (VLA) models have recently demonstrated strong performance in goal-oriented robotic manipulation. However, most existing approaches implic- 1 MBZUAI.Correspondenceto:TongliangLiu <tliang.liu@gmail.com>. Preprint. March 19, 2026. Front Date: 29, Oct Progress Summary: (a) Vanilla VLA Inattentiveness to the label orientation of thewine bottle (b) Kinematics-AwareVLA (Ours) Back Left Right Figure 1. Comparison of Vanilla VLAs vs. KineVLA. Vanilla VLAs (Kim et al., 2025; Wang et al., 2025), which accept coarse command inputs (e.g., “place the wine bottle on the drawer”) and produce relatively fixed bottle label orientation actions. In contrast, Our KineVLA can processes fine kinematic commands (e.g., “control the wine bottle to face a specific orientation on the cabinet”) and generate robot end-effector actions oriented front, back, left, or right. itly assume goal equivalence under instruction paraphrasing, meaning that instructions describing the same task goal can be executed with interchangeable motion trajectories. As a result, these models are largely insensitive to fine-grained kinematic specifications expressed in natural language. In real-world human robot interaction, this assumption fre- quently breaks down. Semantically equivalent goals may require distinct and non-interchangeable trajectories due to explicit kinematic constraints such as position, orientation, distance, and motion direction. For example, as shown in Figure 1, instructions such as “place the bottle on the drawer” and “place the bottle on the drawer with the label facing right, front, or back” describe an identical task goal, yet they demand fundamentally differ- ent execution trajectories and end-effector motions. These trajectories cannot be treated as interchangeable refinements of the same action without violating the instruction seman- tics. This setting goes beyond conventional fine-grained goal specification and instead characterizes a class of tasks that are goal invariant but kinematics variant. Such tasks are not captured by the implicit assumptions underlying most existing VLA formulations. 1 arXiv:2603.17524v1 [cs.RO] 18 Mar 2026 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Figure 2. A task example from our proposed Kinematics-Rich Datasets: In contrast to coarse task instructions that aim to complete a general goal, our approach captures diverse, fine-grained kinematic variations for an action instruction and their temporal evolution across multiple key action stages. These variations encompassA Object Part,B Action Constraint, andC Target Relations, with corresponding images of key actions displayed from left to right. Our KineVLA method is designed to address this challenge, excelling at perceiving these multi-faceted details to achieve precise manipulation. Despite the rapid progress in recent VLA models such as OpenVLA (Kim et al., 2025) andπ 0 (Black et al., 2024), most existing approaches focus on semantic-level ground- ing, mapping visual observations and language descriptions to discrete action tokens or low-frequency control trajecto- ries. While these models generalize well to diverse manip- ulation tasks, their generated actions typically reflect task goal rather than motion precision. In other words, current VLAs are optimized to determine what to do (e.g., grasp or place) but not how to do it (e.g., at which position, distance, or orientation). Moreover, although tokenization-based or diffusion-based decoders improve temporal smoothness, they remain agnostic to fine-grained kinematic cues in lan- guage. This gap prevents existing policies from achieving physically consistent, user-customized manipulations that depend on explicit kinematic instructions. Naive binning struggles with complex actions, while diffusion-based policies are computationally expensive. We therefore adopt vector-quantized action representations (Lee et al., 2024; Wang et al., 2025; Mete et al., 2024; Pertsch et al., 2025) to capture fine-grained motion and temporal consistency for more stable, sample efficient training. To avoid the entanglement of task identity and kinematic re- alization in single-level tokenizers, we further introduce a bi-level action representation that decouples goals and kinematics, making it well suited for kinematics-rich tasks. In order to achieve greater sensitivity to sophisticated fine- grained kinematic actions while preserving robustness to coarse-grained actions, KineVLA proposes a bi-level vector- quantized action discretization scheme, which decomposes robot actions into two complementary latent spaces: a goal- level codebook that captures semantic goals and task goal, and a kinematics-level codebook that encodes precise mo- tion parameters such as direction, distance, and velocity. Bi-level action tokens structure the action space but do not by themselves guarantee correct grounding of kinematic language. We therefore introduce bi-level reasoning tokens as explicit intermediate variables to align instruction-level constraints with hierarchical action representations. Bi-level reasoning tokens parse the task goal and kinematic details of the command. Thus resulting in a bi-level chain-of-thought (CoT)-style generation paradigm, in which the model jointly generates textual reasoning (Reasoning) and discrete action tokens (Action). To ensure consistency between reason- ing and control, we maximize the conditional mutual in- formationI(Reasoning; Action| C), which encourages the generated trajectory to remain semantically grounded, thus forming a coherent link between reasoning and execution. Our reasoning tokens are not intended to improve abstract planning ability, but to serve as explicit, supervised interme- diate variables that align language and action. To alleviate the data scarcity of this new task, we constructed three Kinematics-Rich Datasets that systematically cap- tures a variety of goals and multi-level kinematic details, 2 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition covering diverse tabletop organization and object manipu- lation scenarios. The total number of frames are 378K. We collected paired VLA data using the LIBERO platform (Liu et al., 2023) and the Realman-75 robot. The kinematics-rich language instructions are also annotated with reasoning texts at two levels: 1) general coarse-grained goal, and 2) fine- grained kinematics. Our kinematics-rich VLA datasets cap- ture the entire process of action execution, not only coarsely or partially, as illustrated in Figure 2. The dataset encom- passes diverse actions that encode object-part interactions, target spatial relations, and kinematic action constraints. Such a detailed annotation scheme enables the model to learn highly precise motion representations. Extensive experiments show that KineVLA achieves state- of-the-art performance on kinematics-aware benchmarks. Unlike previous models that produce relatively rigid, pre- defined, and goal-directed motions, our model performs flexible, interpretable, and kinematics-aware operations di- rectly from kinematics-rich instructions. In summary, our main contributions are as follows: •We introduce a new kinematic-rich manipulation task, which bridges semantic goals with diverse action exe- cution trajectories, with the aim of meeting the needs of users’ for customized table organization tasks. •We present KineVLA, a VLA model with bi-level vec- tor quantized action representation that decouples task goals from kinematic variations, enabling interpretable and kinematically sensitive control. •We construct three new Kinematics-Aware Manip- ulation Dataset covering both simulated and real- world scenarios, with detailed spatial, object-level, and motion-constraint annotations to facilitate kinematic reasoning. 2. Related Work 2.1. Vision-Language-Action Models In recent years, VLA models have rapidly progressed from early end-to-end robotic policies grounded in paired vi- sual perception and text to generalist systems capable of long-horizon, multi-stage manipulation. Works such as RT- 2 (Zitkovich et al., 2023) and RT-X (O’Neill et al., 2024) demonstrated that grounding large vision–language models in real robot trajectories can yield surprisingly strong zero- shot control across tasks and embodiments. OpenVLA (Kim et al., 2025) established an influential open-source baseline by coupling a 7B LLM with strong visual encoders and nearly a million real robot demonstrations, showing compet- itive, cross-embodiment generalization, and efficient task adaptation. More recently, theπ 0 (Black et al., 2024) family reframed action generation with flow-matching and high- frequency action chunking for dexterous control, yielding ro- bust closed-loop visuomotor skills from mixed supervisory signals. Building on this,π 0.5 (Intelligence et al., 2025b) co-trains across heterogeneous data sources—multi-robot logs, web-scale semantics, and auxiliary prediction tasks to widen open-world generalization in the wild. Based on their action decoding strategies, current VLA mod- els can be broadly categorized into three classes: those em- ploying per-timestep binning scheme (e.g., (Brohan et al., 2022; Zitkovich et al., 2023; Kim et al., 2025)), diffusion- based decoding (e.g., (Liu et al., 2024; 2025; Black et al., 2024; Intelligence et al., 2025b)), and those adopting vector- quantized action representations (e.g., (Belkhale & Sadigh; Lee et al., 2024; Mete et al., 2024; Pertsch et al., 2025; Wang et al., 2025)). In our work, we emphasize the structural na- ture of action primitives and their compositional semantics. To this end, we adopt vector-quantized action representa- tions approach and introduce a bi-level RVQ-VAE training framework, which jointly captures global task context and fine-grained kinematic patterns, enabling temporally coher- ent and semantically grounded action decoding. 2.2. Physically Reliable Robot Manipulation As the community increasingly seeks to enhance robots’ ability to interact reliably with the real physical world, phys- ically plausible robot manipulation has emerged as a key re- search frontier. Instead of relying solely on visual imitation or simulation priors, recent works integrate force–torque sensing (Yu et al., 2025a; He et al., 2025; Huang et al., 2025), tactile feedback (Huang et al., 2025; Wu et al., 2025; Bi et al., 2025), and affordance-guided (Xu et al., 2024) rea- soning to better capture the dynamics of contact and object interaction. ForceVLA (Yu et al., 2025a) and FoAR (He et al., 2025) augment VLA pipelines with wrench data and impedance feedback, improving compliance, force regulation, and fine contact control in real manipulation. Tactile-VLA (Huang et al., 2025) and VLA-Touch (Bi et al., 2025) demonstrate the benefit of integrating high-frequency tactile embeddings and visuotactile fusion, enabling policies that can infer slip, deformation, and surface geometry directly from touch sig- nals. FreeTacMan (Wu et al., 2025) further scales tactile- conditioned manipulation across diverse objects and sensors through self-supervised visuotactile representation learn- ing. In parallel, affordance-driven models such as Natu- ralVLM (Xu et al., 2024) bridge semantic perception and physical reasoning by predicting object parts, functional regions, and graspable affordances conditioned on natural- language queries. Overall, these efforts can be viewed as emphasizing physical dynamics through the incorporation of additional sensory 3 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition modalities. In contrast, our work focuses on fine-grained, user-customized instruction tasks, highlighting kinematic- level reasoning of robot motion under diverse physics-aware settings, thereby bridging linguistic goal and physically grounded, precise action execution. 3. Method In this section, we present our method, comprising a bi- level RVQ-VAE and KineVLA. We begin by presenting bi-level vector quantized action representation (Sec. 3.1), detailing its implementation specifics of training a Bi-Level Residual VQ-VQE. We then describe the bi-level generation paradigm and the associated mutual-information regulariza- tion scheme (Sec. 3.2). 3.1. Bi-Level Action Representation Uniform binning of each action dimension, as in OpenVLA- style discretization where token IDs lie in[0, 255], fixes par- tition boundaries and ignores cross-joint correlations. This affects the robustness of low-frequency trends and sensitiv- ity to high-frequency transients. We instead learn discrete vector-quantized (VQ) action tokens inspired by (Zeghidour et al., 2021; Lee et al., 2024; Wang et al., 2025), then extend them to a bi-level design that targets both regimes. Vector-Quantized Action Representation. Given a chunk of an action sequenceA t:t+H ∈R T×D , an encoderf θ pro- duces latentsz t = f θ (A t:t+H )∈R d . It usesMcodebooks E (m) =e (m) k K m k=1 applied sequentially to quantify residu- als by Residual Vector Quantized Variational AutoEncoder (RVQ-VAE) (Zeghidour et al., 2021). Initializer (0) t = z t and for m = 1,...,M : k (m) t = arg min k r (m−1) t − e (m) k 2 2 , ̃z (m) t = e (m) k (m) t , r (m) t = r (m−1) t − ̃z (m) t . (1) The final quantized latent and discrete token tuple are ̃z t = M X m=1 ̃z (m) t ,i t = k (1) t ,...,k (M ) t (tokens). (2) A decoderg φ reconstructs continuous actions ˆ A t:t+H = g φ ( ̃z 1:T ). We train with a reconstruction term and RVQ codebook and commitment terms: L rec = 1 T T X t=1 ∥A t − ˆ A t ∥ 1 , L rvq =L rec + M X m=1 T X t=1 sg(r (m−1) t )− ̃z (m) t 2 2 + M X m=1 T X t=1 β m r (m−1) t − sg( ̃z (m) t ) 2 2 , (3) wheresg(·)is the stop-gradient operator andβ m balances commitment at levelm. Unlike OpenVLA’s binning scheme, our partitions are learned, non-uniform, and compositional acrossMresidual levels, which couples joints and time while yielding compact semantic tokens. Bi-Level Action Representation. Building on the above vector-quantized action representation formulation, we fac- tor actions into two discrete spaces backed by two code- books of identical capacity: a goal (coarse-grained) level that captures low-frequency structure and task goal, and a kinematics (fine-grained) level that captures high-frequency corrections and contact-rich transients. LetE l ∈R K×d andE h ∈R K×d denote the goal and kinematics codebooks (same asK,d), which are then combined and fed to the decoder to reconstruct the action chunk. Bi-Level RVQ-VAE Training. We adopt a two-stage train- ing schedule with stepwise codebook learning, while keep- ing the two codebooks of equal size. This design explicitly encourages the fine-grained codebook to capture kinematics- sensitive action representations, enabling it to model subtle action variations more effectively: Stage I (pretraining with coarse-grained actionsA G t:t+H ). Train the coarse path with RVQ-VAE on broad, multi-task data (e.g., Open X-Embodiment (O’Neill et al., 2024), Man- iSkill (Mu et al., 2021)) to learnE l and its quantizer. This stage encourages temporally stable, low-frequency tokens. Stage I (finetuning with kinematics-aware actions A K t:t+H ). FreezeE l and the coarse-grained codebook quan- tizer. Initialize a residual path with a new RVQ stack and codebookE h of the same size, warm-start the shared en- coder/decoder from Stage I, and train on Kine Datasets to specializeE h for high-frequency details. OnlyE h and the fine-grained codebook quantizer are learned from scratch; shared modules are finetuned. 3.2. Bi-Level Generation After decoupling actions into a two-level representation, we predict bi-level reasoning tokens to align internal represen- tations with language parsing. This is inspired by chain-of- thought in large language models (Yang et al., 2025; Guo et al., 2025), this design improves interpretability, constraint 4 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Figure 3. Overview of our framework. Our approach decouples low-frequency, goal-level control from fine-grained kinematic refinements to effectively handle kinematics-rich tasks. (a) The proposed Bi-Level RVQ-VAE learns hierarchical action representations (Sec. 3.1), while (b) the KineVLA framework addresses kinematics-rich tasks through bi-level generation (Sec. 3.2). injection and is trained end to end via supervised finetun- ing, avoiding the instability of reinforcement learning (Yu et al., 2025b). To improve coherence between reasoning and action tokens, we introduce a mutual information regu- larization that aligns the two bi-level streams and reduces execution drift. Reasoning Tokens Prediction. Given the input imageI t and a kinematics rich instruction T kine , we fine-tuning an end to end base VLM modelf θ to produce two reasoning text outputst coarse andt fine . The coarse reasoning text expresses the general task goal in natural language, while the fine reasoning text specifies key kinematics parameters and anchor points; see Section. 4 for concrete examples. Training uses a language cross entropy objective over both outputs, L reasoning (θ) = X m∈c,f L m X t=1 CE h m t , p θ h m t | h m <t , C , (4) whereC = (I t , t kine ),h c t andh f t are tokens from the coarse and fine reasoning texts, L c and L f are their lengths. Action Tokens Prediction. We use the bi-level action rep- resentation derived in Sec. 3.1 as ground-truth labels for supervised prediction. Because the action codebook indices learned by the bi-level RVQ-VAE reside in their own in- dex space, we offset these indices to the tail of the text vocabulary. This lets the model reuse a shared output head and apply a unified loss across both text and action tokens, avoiding any extra task-specific head while keeping super- vision consistent. Following the reasoning text prediction formulation, the prediction objective is L action (θ) = X m∈c,f L m X t=1 CE a m t , p θ a m t | a m <t , C ,(5) wherea c t anda f t are tokens from the coarse-grained and fine-grained action tokens. Mutual Information Regularization for reason Text and Action. Although the proposed model jointly gen- erates textual reasoning (Reasoning) and continuous control trajectories (Action), these two objectives are often opti- mized independently, which can lead to semantic discon- nection. In such cases, textual reasoning may not faith- fully reflect the underlying motor intention. To address this issue, we maximize conditional mutual information I(Reasoning; Action| C), whereCdenotes the given con- text (e.g., task condition or observation). We achieve this by pulling paired embeddings(Reasoning, Action)closer together and pushing mismatched pairs apart using an In- foNCE objective (Oord et al., 2018; He et al., 2020; Chen et al., 2020). Formally, given a batch(T i ,A i ,C i ) N i=1 , we obtain em- beddingst i = f T (T i ,C i )anda i = f A (A i ,C i ), and define the similarity s ij = t ⊤ i a j τ ,(6) where τ > 0 is a temperature parameter. The symmetric mutual information regularization loss is L MI =− 1 2N N X i=1 " log e s i P N j=1 e s ij + log e s i P N j=1 e s ji # .(7) 5 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Algorithm 1 Bi-Level Action Coupling Inference Require:Sequence lengthT, chunk sizeH=5; observa- tions(I t , W t ) T t=1 ; kinematic instructionT kine ; gen- eratorG; bi-level decoderD Ensure: Predicted action chunks ˆ A t:t+H T−H t=1 1: for t = 1 to T − H do 2:o t ← Enc obs (I t , W t ) 3:t kine ← Enc kin (T kine ) 4: C t ← BuildContext(o t , t kine ) 5: (h c t , a c t ), (h f t , a f t )←G(C t ) # Bi-level generation 6:a ⋆ t ← [ a c t ; a f t ] 7: ˆ A t:t+H ←D(a ⋆ t ) # Chunked action decoding 8: end for Bi-Level Action Coupling Inference. As presented in Al- gorithm 1 , we perform action inference from a scene image, a wrist image, and a kinematic instruction using a generator– decoder pipeline. The generator emits bi-level chain-of- thought text and discrete action codebooks: a high-level plan and a low-level control trace. Each level is delimited with special tokens and paired with its own action codes. A bi-level decoder then fuses both codebooks to reconstruct a single action chunk. 3.3. The Base Vision-Language Model We build KineVLA on top of OpenVLA (Kim et al., 2025) as the base vision–language. Whereas OpenVLA origi- nally adopts a discrete action tokenization scheme, we re- place it with our proposed bi-level action representation. The action representation are learned via a residual VQ- VAE (Zeghidour et al., 2021) trained for action reconstruc- tion, enabling compact yet expressive control sequences. We retain OpenVLA’s Prismatic-7B (Karamcheti et al., 2024) vision–language backbone, pre-trained on large-scale im- age–text data, and perform a second-stage adaptation using LoRA (rank = 32) applied to all linear layers, with Gaus- sian initialization of the LoRA weights. The input pipeline mirrors OpenVLA except that the natural-language instruc- tion channel is replaced by fine-grained kinematics; other inputs (e.g., visual observations and task context) remain unchanged. The decoder jointly emits reason text and action tokens, both mapped into the same vocabulary index space as text, facilitating the use of a unified entropy loss. Unless otherwise noted, we adhere to OpenVLA’s standard data preprocessing, and optimization hyperparameters. 4. Experiments We assess the effectiveness of our proposed approach through a series of experiments conducted on both simula- tion benchmarks and real-world robotic manipulation tasks. These experiments are designed to answer the following research questions: • Whether our method provides clear advantages over diffusion-based models and single-level VQ-VAE paradigms in kinematics-rich manipulation tasks. •Does it achieve strong performance in both simulation and real-world robot settings? •How much does the inclusion of chain-of-thought rea- soning contribute to overall model performance? • How effective is mutual information optimization in improving the prediction of reason text and action out- puts? 4.1. Experimental Setup We leverage three datasets (LIBERO-Goal-Relabeled, LIBERO-GOAL, Kine-Realman) to comprehensively cover a wide variety of goals and to annotate them with diverse, fine-grained kinematics-rich instructions. For details, please refer to Table 4 in Appendix for a summary of our proposed dataset. Additional details are provided in the Appendix. All kinematics-related language descriptions in our datasets are defined in a consistent third-person camera coordinate frame aligned with human observation. Directional and positional terms such as left, right, near, and far are spec- ified relative to the external camera view, rather than the robot base or end-effector frame. For each task, we curate a total of 50 demonstrations and assess performance across 20 evaluation trials. RVQ-VAE is trained on a single A100 GPU with a batch size of 1024. For a fair comparison, all fine-tuning is conducted for 50K gradient steps with a total batch size of 64, using 4 A100 GPUs and an action chunk length of K=5. Evaluation Metrics. For reasoning evaluation, we assess quality using three complementary metrics. BERTScore measures contextual semantic similarity and is robust to paraphrasing, while BLEU reports n-gram overlap as a surface-level metric. In addition, we include an Intervention metric, which replaces the predicted reasoning tokens with either randomly sampled reasoning tokens or mismatched reasoning tokens drawn from other instructions. Baselines. We compare with OpenVLA (Kim et al., 2025), π 0.5 (Intelligence et al., 2025a), and VQ-VLA (Wang et al., 2025). OpenVLA is a vision–language–action transformer that conditions on image features and a text goal to regress continuous low-level actions.π 0.5 is a diffusion-based model, which uses co-training on heterogeneous tasks to enable broad generalization. VQ-VLA discretizes the action space using a single-level vector-quantized action tokenizer. For fairness, these methods consume the same vision obser- vations and the same kinematics-rich language instructions. 6 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Figure 4. Experimental Results and Comparisons. We benchmark our method across the three proposed kinematics-aware datasets, encompassing both simulation and real-world robotic experiments. The left two figures illustrate example environments, while the bar chart on the right presents the goal and kinematic success rates. 4.2. Evaluations Results Quantivative Performance Analysis. As shown in Fig- ure 4, all methods achieve comparable performance in terms of goal success rate across both simulation and real- world benchmarks, with only marginal differences between models. This indicates that existing VLA approaches are generally effective at capturing high-level task objectives. In contrast, when evaluated on kinematics success rate, the performance gap becomes substantially larger. While base- line methods exhibit a notable drop in kinematic accuracy, KineVLA consistently achieves significantly higher success rates across all datasets. This discrepancy highlights a key limitation of prior ap- proaches, which tend to collapse fine-grained kinematic vari- ations once the task goal is reached. By contrast, KineVLA explicitly decouples goal-level intent from kinematics-level execution through its bi-level action representation and structured reasoning alignment, enabling precise adherence to instruction-level kinematic constraints. These results demonstrate that modeling kinematics as a first-class compo- nent is essential for kinematics-rich tasks, and that improve- ments in such settings cannot be achieved by goal-oriented modeling alone. Visualization Examples. Figure 5 illustrates qualitative examples of task execution using KineVLA, highlighting its ability to reason and act effectively in kinematics-aware ma- nipulation tasks. From left to right, the figure demonstrates how KineVLA processes a kinematics-rich instruction, inter- prets the initial environmental state, and generates bi-level reasoning text, comprising goal-level and kinematics-level reasoning. The corresponding bi-level action tokens are then decoded into concrete robot motions, shown through key intermediate states and the final successful execution. This hierarchical reasoning framework enables KineVLA to bridge high-level semantic understanding with precise low-level motion control. For instance, it explicitly plans object contact, trajectory, and pose orientation, ensuring stability and accuracy during manipulation. Bi-level Reasoning Analysis. Table 1 shows that KineVLA achieves consistently high BLEU scores across all datasets, indicating strong alignment between the generated bi-level reasoning texts and the ground-truth annotations. To assess the necessity of bi-level reasoning, we conduct intervention- based analysis by replacing the predicted reasoning tokens with randomly sampled or mismatched ones while keep- ing visual inputs and task goals unchanged. This leads to a substantial drop in kinematics-following success rates from 76.5% to 52.4%, from 70.4% to 48.6%, and from 65.0% to 42.4% across the three datasets, respectively, while goal completion remains largely unaffected. These results demonstrate that bi-level reasoning tokens play a causal role in grounding kinematic constraints into action generation. Inference Speed Analysis. As shown in Table 2, KineVLA does not introduce a significant inference overhead com- pared to single-level VQ-VAE models and diffusion-based methods. While slightly slower than VQ-VLA andπ 0.5 , the inference time of KineVLA remains in a similar range. This efficiency is achieved because the bi-level design op- erates at the feature and token level, requiring only a small increase in output tokens rather than additional decoding stages, resulting in minimal impact on overall inference speed. 7 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Figure 5. Task execution examples using KineVLA. From left to right, the figure shows the kinematics-rich instruction input, the initial state of the environment, followed by the bi-level reasoning text and action tokens generated by the KineVLA model. Next are two columns illustrating the key robot action states, and finally, the resulting final state after the robot executes the task. Table 1.Bi-level reasoning evaluation using the BLEU, BERTScore and Intervention metrics. DatasetBLEUBERTScoreIntervention (%) LIBERO-Goal-Relabled0.930.9476.5→52.4 Kine-LIBERO0.950.9670.4→48.6 Kine-Realman0.920.9465.0→42.4 Table 2. Inference Speed Comparison. MethodsInference Time OpenVLA237ms π 0.5 86ms VQ-VLA84ms KineVLA96ms 4.3. Ablation Study The ablation results in Table 2 show the contribution of each proposed component in KineVLA. Starting from the Baseline, which uses standard vector-quantized action to- kenization with OpenVLA, introducing Bi-Rep leads to a clear improvement in performance, increasing success rates from 70.0% to 73.4% on LIBERO-Goal-Relabeled and from 61.5% to 66.8% on Kine-LIBERO. This demonstrates that the bi-level action representation effectively captures finer- grained motion semantics. Adding Bi-Rea, which introduces bi-level reasoning tokens, further improves performance to 75.5% and 68.6% on the two datasets. This demonstrates the importance of explicitly aligning language instructions with bi-level action represen- Table 3. Ablation studies of KineVLA components. Success rates (%) across two kinematics-rich datasets on LIBERO platform. “Baseline” refers to the standard vector-quantized action tokeniza- tion using OpenVLA. “+ Bi-Rep” denotes the use of our Bi-Level action representation. “+ Bi-Rea” indicates the incorporation of bi-level reasoning token. “+ MI” refers to the addition of mutual information optimization between the reasoning text and the ac- tion. DatasetBaseline+ Bi-Rep+ Bi-Rea+ MI LIBERO-Goal-Relabled70.073.475.576.5 Kine-LIBERO61.566.868.670.4 tations, rather than relying on action modeling alone. Finally, integrating MI, the mutual information optimization between the reasoning text and the action, achieves the best results of 76.5% on LIBERO-Goal-Relabeled and 70.4% on Kine-LIBERO. This shows that aligning textual reasoning with action execution leads to more consistent and robust behavior. Overall, these results confirm the effectiveness and comple- mentarity of the proposed bi-level representation, reasoning, and optimization modules. 5. Conclusion and Limitation In this work, we introduce KineVLA, a kinematics-rich vision–language–action framework that enables robots to understand and execute fine-grained motion instructions by explicitly disentangling kinematic sensitivity from goal 8 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition invariance. This design allows a single goal to be real- ized through actions with varying kinematic granularity, improving both flexibility and interpretability. To support this task, we construct comprehensive simulated and real- world datasets with detailed kinematic annotations. Exten- sive experiments on tabletop manipulation demonstrate that KineVLA achieves superior precision and generalization compared to existing VLA models. While our current eval- uation is limited to tabletop settings, the proposed bi-level formulation is naturally extensible to whole-body manipula- tion with more complex kinematic dependencies, which we leave for future work. References Belkhale, S. and Sadigh, D.Minivla: A better vla with a smaller footprint, 2024.URL https://github. com/Stanford-ILIAD/openvla-mini. Bi, J., Ma, K. Y., Hao, C., Shou, M. Z., and Soh, H. Vla-touch: Enhancing vision-language-action mod- els with dual-level tactile feedback.arXiv preprint arXiv:2507.17294, 2025. Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al. pi0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In International conference on machine learning, p. 1597–1607. PmLR, 2020. Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. Deepseek- r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638, 2025. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Mo- mentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9729–9738, 2020. He, Z., Fang, H., Chen, J., Fang, H.-S., and Lu, C. Foar: Force-aware reactive policy for contact-rich robotic ma- nipulation. IEEE Robotics and Automation Letters, 2025. Huang, J., Wang, S., Lin, F., Hu, Y., Wen, C., and Gao, Y. Tactile-vla: Unlocking vision-language-action model’s physical knowledge for tactile generalization. arXiv preprint arXiv:2507.09160, 2025. Intelligence, P., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., et al.pi0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025a. Intelligence, P., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., et al.pi0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025b. Karamcheti, S., Nair, S., Balakrishna, A., Liang, P., Kol- lar, T., and Sadigh, D. Prismatic vlms: Investigating the design space of visually-conditioned language mod- els. In Forty-first International Conference on Machine Learning, 2024. Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakr- ishna, A., Nair, S., Rafailov, R., Foster, E. P., Sanketi, P. R., Vuong, Q., et al. Openvla: An open-source vision- language-action model. In Conference on Robot Learning, p. 2679–2713. PMLR, 2025. Lee, S., Wang, Y., Etukuru, H., Kim, H. J., Shafiullah, N. M. M., and Pinto, L. Behavior generation with latent actions. arXiv preprint arXiv:2403.03181, 2024. Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., and Stone, P. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791, 2023. Liu, J., Chen, H., An, P., Liu, Z., Zhang, R., Gu, C., Li, X., Guo, Z., Chen, S., Liu, M., et al. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language- action model. arXiv preprint arXiv:2503.10631, 2025. Liu, S., Wu, L., Li, B., Tan, H., Chen, H., Wang, Z., Xu, K., Su, H., and Zhu, J. Rdt-1b: a diffusion founda- tion model for bimanual manipulation. arXiv preprint arXiv:2410.07864, 2024. Mete, A., Xue, H., Wilcox, A., Chen, Y., and Garg, A. Quest: Self-supervised skill abstractions for learning continuous control. Advances in Neural Information Processing Sys- tems, 37:4062–4089, 2024. Mu, T., Ling, Z., Xiang, F., Yang, D., Li, X., Tao, S., Huang, Z., Jia, Z., and Su, H. Maniskill: Generalizable manipu- lation skill benchmark with large-scale demonstrations. arXiv preprint arXiv:2107.14483, 2021. Oord, A. v. d., Li, Y., and Vinyals, O. Representation learn- ing with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 9 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition O’Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 6892–6903. IEEE, 2024. Pertsch, K., Stachowicz, K., Ichter, B., Driess, D., Nair, S., Vuong, Q., Mees, O., Finn, C., and Levine, S. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025. Wang, Y., Zhu, H., Liu, M., Yang, J., Fang, H.-S., and He, T. Vq-vla: Improving vision-language-action models via scaling vector-quantized action tokenizers. arXiv preprint arXiv:2507.01016, 2025. Wu, L., Yu, C., Ren, J., Chen, L., Jiang, Y., Huang, R., Gu, G., and Li, H. Freetacman: Robot-free visuo-tactile data collection system for contact-rich manipulation. arXiv preprint arXiv:2506.01941, 2025. Xu, R., Shen, Y., Li, X., Wu, R., and Dong, H. Naturalvlm: Leveraging fine-grained natural language for affordance- guided visual manipulation. IEEE Robotics and Automa- tion Letters, 2024. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yu, J., Liu, H., Yu, Q., Ren, J., Hao, C., Ding, H., Huang, G., Huang, G., Song, Y., Cai, P., et al. Forcevla: Enhanc- ing vla models with a force-aware moe for contact-rich manipulation. arXiv preprint arXiv:2505.22159, 2025a. Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025b. Zeghidour, N., Luebs, A., Omran, A., Skoglund, J., and Tagliasacchi, M. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507, 2021. Zitkovich, B., Yu, T., Xu, S., Xu, P., Xiao, T., Xia, F., Wu, J., Wohlhart, P., Welker, S., Wahid, A., et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, p. 2165–2183. PMLR, 2023. 10 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition A. Datasets and Visualization LIBERO (Simulation).Build on LIBERO platform (Liu et al., 2023), operators teleoperate the end-effector with a SpaceMouse. For each rollout, we record time-stamped RGB and wrist frames, joint states, forward-kinematics end-effector pose, and the action stream. We adopt the same preprocessing pipeline as in (Liu et al., 2023), which includes: (1) removing idle segments from the trajectories, (2) resizing all images to a resolution of 256×256 pixels, and (3) rotating each image by 180 degrees. We build two splits: •LIBERO-Goal-Relabeled: LIBERO-GOAL covers 10 tasks and focuses on scenarios where the robot needs to continually acquire new knowledge about motions and behaviors. We retain the visual observations and action trajectories from LIBERO-GOAL and relabel the language instructions into their corresponding kinematics-rich versions based on human observation, making each action more specific. For example, “put the wine bottle on the rack” is relabeled as “grab the wine bottle by the neck and place it at the front of the rack.” More details can be found in the supplementary material. •Kine-LIBERO (ours): We build upon the LIBERO platform and collect a larger set of kinematics-rich tasks in table manipulation scene, consisting 266K frames in total. Each episode is annotated with kinematics-rich language instructions and bi-level reasoning texts (goal and kinematics). As illustrated in Figure 2, our dataset features variations across the entire timeline, with diverse object parts, action constraints, and target relations. Further details are provided in the supplementary material. Realman-75 (Real Robot). We construct the Kine-Realman dataset using a Realman-75 robotic arm equipped with a parallel gripper, a calibrated RGB camera observing a fixed tabletop workspace, and an additional wrist-mounted camera. The robot is teleoperated via puppet manipulation, and we collect data for seven tasks featuring rich, fine-grained kinematic descriptions. For example, one instruction specifies “grasp the lower half of the carrot and place it on the left side of the plate.” Further details about the tasks are provided in the supplementary material. Visualization. As detailed in Tables 7,6, and 5, we have enumerated text examples from our annotated dataset alongside their corresponding bi-level Chain-of-Thought (CoT) annotations. Figure 6 and 7 illustrates the execution outcomes of our method on the Kine-LIBERO and Kine-Realman-75 benchmarks. The results from both benchmarks confirm that our method achieves precise execution of fine-grained kinematics operations in a consistent environment. Figure 6 demonstrates the executed tasks, categorized into two groups. The first group (Tasks 7-10) is concerned with opening drawers of different layers to various degrees (completely, halfway, slightly). The second group (Tasks 12,14) focuses on moving a plate following distinct trajectories (leftward then backward, diagonally toward the back-right). Figure 7 demonstrates the successful execution of two distinct task sets. The first set (Tasks 1, 3) demonstrates relocation of a piece of bread to different specified locations. The second set (Tasks 4-7) shows the grasping of a carrot at different parts (top/bottom) and its placement into different areas of a plate. Table 4. Summary of our proposed datasets collected via SpaceMouse and puppet manipulation. In each task, we gather 50 demonstrations. SplitSettingFramesLanguageReasoning TextVisionAction Space LIBERO-Goal-RelabeledSimulation63KKinematics-richBi-LevelFront, WristEEF Pose Kine-LIBEROSimulation266KKinematics-richBi-LevelFront, WristEEF Pose Kine-Realman-75Real robot49KKinematics-richBi-LevelFront, WristJoint Positions 11 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Figure 6. Execution results on the Kine-LIBERO dataset. The serial number corresponds to the entry in Table 6. 12 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Figure 7. Execution results on the Kine-Realman-75 dataset. The serial number corresponds to the entry in Table 7. 13 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Table 5. LIBERO-Goal-Relabeled Dataset. No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 1grasp the left edge of the bowl, move it forward, and place it precisely at the center of the plate put the bowl on the plate1. Robot Action-level Constraint: Trajec- tory shape - forward linear movement; Con- tact timing - grasp then place; Ordinal re- lations - sequential grasp, move, place. 2. Object-level Targeting: Target - bowl’s left edge for grasp, plate’s center for placement; Pose/Facing/Angle - precise placement with implied alignment. 2 grab the wine bottle by the neck and place it on the rack put the wine bottle on the rack1. Robot Action-level Constraint: Con- tact Timing - Must initiate/break contact (grab/release). Trajectory Shape - Requires transfer motion from source (neck) to target (rack). Ordinal Relations - Strict sequence: grab before place. 2. Object-level Targeting: Grasp Target - The neck (specific part) of wine bottle. Place Target - The rack (spe- cific object); final pose constrained by rack structure (must be placed on it). 3press the handle to open the top drawer to its maximum extension, then grasp the left edge of the bowl and place it in the center of the drawer open the top drawer and put the bowl inside1.Robot Action-level Constraint: State/Trajectory:Press,thengrasp and place. Contact Timing: Press handle first, then grasp bowl after drawer is open. Ordinal Relations: Sequential: open drawer, then grasp bowl, then place bowl. 2. Object- level Targeting:Handle/Surface/Edge: Handle (of drawer), left edge (of bowl). Pose/Facing/Angle: Maximum extension (drawer state), center (target placement pose). 4grab the cream cheese by the middle and place it in the center of the bowl put the cream cheese in the bowl1. Robot Action-level Constraint: Trajectory shape - Straight-line placement motion; Con- tact timing - Grasp first, then place; Ordi- nal relations - Sequential (grab, then place). 2.Object-level Targeting: Handle/Surface - Target middle of object (cream cheese); Pose/Facing - Orientation-agnostic grasp; Placement - Center of bowl (specific posi- tional target). 5hold the wine bottle firmly by the neck and position it on the upper section of the cabinet top, orienting the bottle so that its label is facing to the left put the wine bottle on top of the cabinet1.Robot Action-level Constraint: State/Trajectory:Hold firmly,posi- tion on upper section. Contact Timing: Grip neck before positioning. Ordinal Relations: Orient after positioning. 2. Object-level Tar- geting: Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle: Label facing left. 6place the end-effector at the center of the plate and move it leftward then backward along the table plane to repo- sition the plate about close in front of the stove push the plate to the front of the stove1.Robot Action-level Constraint: State/Trajectory:Moveend-effector leftward then backward along the table plane.Contact Timing: Maintain con- tact with the plate.Ordinal Relations: Sequential motion: first leftward, then backward. 2. Object-level Targeting: Target Part: Center of the plate (initial contact). Pose/Facing/Angle: Align end-effector to plate center; movement constrained to table plane. Final Placement: close in front of the stove. Continued on next page 14 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 7move left to the stove knob, then turn it clockwise to ignite turn on the stoveRobot Action-level Constraint: Trajectory shape - Linear translation (move left) fol- lowed by rotational (turn clockwise). Con- tact timing - Sequential (move then turn). Ordinal relations - Strict order: translation before rotation. Object-level Targeting: Tar- get object part - Stove knob (specific com- ponent). Pose/facing/angle - Clockwise ro- tation (specific angular direction). 8grab the left edge of this bowl and move it along a diagonal path to the center of the stove put the bowl on the stove Robot Action-level Constraint: Trajectory shape - Diagonal path; Contact timing - Grab (initial contact) Object-level Targeting: Target part - Left edge; Object - Bowl; Final pose - Center of the stove. 9 grab the right edge of this bowl and move it along a diagonal path to the upper section of the cabinet put the bowl on top of the cabinet1. Robot Action-level Constraint: Trajec- tory shape - Diagonal path; Contact timing - After grabbing, then move 2. Object-level Targeting: Target component - Right edge of the bowl; Spatial relation - Upper section of the cabinet. 10hold the handle of the middle cabinet drawer and pull it open completely open the middle drawer of the cabinet 1. Robot Action-level Constraint: Contact Timing: Initiate contact Trajectory Shape: Linear pulling Ordinal Relations: Grasp then pull 2. Object-level Targeting: Target Part: Middle cabinet drawer handle Action on Target: Pull open completely. Table 6. Kine-LIBERO Dataset No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 1grab the right edge of this bowl and move it along a diagonal path to the upper section of the cabinet put the bowl on top of the cabinet1. Robot Action-level Constraint: Tra- jectory shape - Diagonal path; Con- tact timing - After grabbing, then move 2. Object-level Targeting: Tar- get component - Right edge of the bowl; Spatial relation - Upper section of the cabinet 2grab the left edge of this bowl and move it along a diagonal path to the upper section of the cabinet put the bowl on top of the cabinet1. Robot Action-level Constraint: Tra- jectory shape - Diagonal path; Con- tact timing - After grabbing, then move 2. Object-level Targeting: Tar- get component - Left edge of the bowl; Spatial relation - Upper section of the cabinet 3grab the right edge of this bowl and move it along a diagonal path to the bottom section of the cabinet put the bowl on top of the cabinet1. Robot Action-level Constraint: Tra- jectory shape - Diagonal path; Con- tact timing - After grabbing, then move 2. Object-level Targeting: Tar- get component - Right edge of the bowl; Spatial relation - Bottom sec- tion of the cabinet Continued on next page 15 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 4grab the left edge of this bowl and move it along a diagonal path to the bottom section of the cabinet put the bowl on top of the cabinet1. Robot Action-level Constraint: Tra- jectory shape - Diagonal path; Con- tact timing - After grabbing, then move 2. Object-level Targeting: Tar- get component - Left edge of the bowl; Spatial relation - Bottom sec- tion of the cabinet 5hold the handle of the middle cabinet drawer and pull it open completely open the middle drawer of the cabinet1.Robot Action-level Constraint: Contact Timing: Initiate contact Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull 2. Object-level Targeting: Target Part: Middle cabinet drawer handle. Ac- tion on Target: Pull open completely 6vertically hold the handle of the mid- dle cabinet drawer and pull it open completely open the middle drawer of the cabinet1.Robot Action-level Constraint: Contact Timing: Initiate contact. Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull. 2. Object-level Targeting: Target Part: Middle cabinet drawer handle. Ac- tion on Target: Pull open completely 7vertically hold the handle of the mid- dle cabinet drawer and pull it open halfway open the middle drawer of the cabinet1.Robot Action-level Constraint: Contact Timing: Initiate contact. Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull. 2. Object-level Targeting: Target Part: Middle cabinet drawer handle. Ac- tion on Target: Pull open halfway 8vertically hold the handle of the mid- dle cabinet drawer and pull it slightly open open the middle drawer of the cabinet1.Robot Action-level Constraint: Contact Timing: Initiate contact. Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull. 2. Object-level Targeting: Target Part: Middle cabinet drawer handle. Ac- tion on Target: Pull open slightly 9vertically hold the handle of the top cabinet drawer and pull it open com- pletely open the top drawer of the cabinet 1.Robot Action-level Constraint: Contact Timing: Initiate contact. Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull. 2. Object-level Targeting: Target Part: Top cabinet drawer handle. Action on Target: Pull open completely 10vertically hold the handle of the top cabinet drawer and pull it open halfway open the top drawer of the cabinet1.Robot Action-level Constraint: Contact Timing: Initiate contact. Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull. 2. Object-level Targeting: Target Part: Top cabinet drawer handle. Action on Target: Pull open halfway Continued on next page 16 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 11vertically hold the handle of the top cabinet drawer and pull it slightly open open the top drawer of the cabinet1.Robot Action-level Constraint: Contact Timing: Initiate contact. Tra- jectory Shape: Linear pulling. Or- dinal Relations: Grasp then pull. 2. Object-level Targeting: Target Part: Top cabinet drawer handle. Action on Target: Pull open slightly 12push the plate and move it leftward then backward along the table plane to reposition the plate about close in front of the stove push the plate to the front of the stove1.Robot Action-level Constraint: State/Trajectory: Move end-effector leftward then backward along the ta- ble plane. Contact Timing: Main- tain contact with the plate.Ordi- nal Relations: Sequential motion: first leftward, then backward.2. Object-level Targeting: Target Part: Center of the plate (initial con- tact). Pose/Facing/Angle: Align end- effector to plate center; movement constrained to table plane.Final Placement: ̃close in front of the stove. 13push the plate and move it rightward then backward along the table plane to reposition the plate about far in front of the stove push the plate to the front of the stove1.Robot Action-level Constraint: State/Trajectory: Move end-effector rightward then backward along the table plane. Contact Timing: Main- tain contact with the plate.Ordi- nal Relations: Sequential motion: first rightward, then backward. 2. Object-level Targeting: Target Part: Center of the plate (initial con- tact). Pose/Facing/Angle: Align end- effector to plate center; movement constrained to table plane.Final Placement: ̃far in front of the stove. 14push the plate and move it diagonally toward the back-right along the ta- ble plane to reposition the plate about close in front of the stove push the plate to the front of the stove1.Robot Action-level Constraint: State/Trajectory: Move end-effector diagonally toward back-right along the table plane.Contact Tim- ing: Maintain contact with the plate. Ordinal Relations: Sequential mo- tion: first diagonally back-right. 2. Object-level Targeting: Target Part: Center of the plate (initial con- tact). Pose/Facing/Angle: Align end- effector to plate center; movement constrained to table plane.Final Placement: ̃close in front of the stove. Continued on next page 17 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 15push the plate and move it diagonally toward the back-right along the table plane to reposition the plate about far in front of the stove push the plate to the front of the stove1.Robot Action-level Constraint: State/Trajectory: Move end-effector diagonally toward back-right along the table plane.Contact Tim- ing: Maintain contact with the plate. Ordinal Relations: Sequential mo- tion: first diagonally back-right. 2. Object-level Targeting: Target Part: Center of the plate (initial con- tact). Pose/Facing/Angle: Align end- effector to plate center; movement constrained to table plane.Final Placement: ̃far in front of the stove. 16push the plate and move it rightward then backward along the table plane to reposition the plate about close in right of the stove push the plate to the right the stove1.Robot Action-level Constraint: State/Trajectory: Move end-effector rightward then backward along the table plane. Contact Timing: Main- tain contact with the plate.Ordi- nal Relations: Sequential motion: first rightward, then backward. 2. Object-level Targeting: Target Part: Center of the plate (initial con- tact). Pose/Facing/Angle: Align end- effector to plate center; movement constrained to table plane.Final Placement: ̃close in right of the stove. 17 push the plate and move it diagonally toward the back-right along the ta- ble plane to reposition the plate about close in right of the stove push the plate to the right the stove1.Robot Action-level Constraint: State/Trajectory: Move end-effector diagonally toward back-right along the table plane.Contact Tim- ing: Maintain contact with the plate. Ordinal Relations: Sequential mo- tion: first diagonally back-right.2. Object-level Targeting: Target Part: Center of the plate (initial con- tact). Pose/Facing/Angle: Align end- effector to plate center; movement constrained to table plane.Final Placement: ̃close in right of the stove. Continued on next page 18 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 18grab the wine bottle by the neck and place it on the rack put the wine bottle on the rack1.Robot Action-level Constraint: Contact Timing - Must initiate/break contact (grab/release).Trajectory Shape - Requires transfer motion from source (neck) to target (rack). Ordinal Relations - Strict sequence: grab before place. 2. Object-level Targeting: Grasp Target - The neck (specific part) of wine bottle. Place Target - The rack (specific object); fi- nal pose constrained by rack structure (must be placed on it). 19grab the wine bottle by the body and place it at the front of the rack put the wine bottle on the rack1.Robot Action-level Constraint: Contact Timing - Must initiate/break contact (grab/release).Trajectory Shape - Requires transfer motion from source (body) to target (rack front). Ordinal Relations - Strict se- quence: grab before place. 2. Object- level Targeting: Grasp Target - The body (specific part) of wine bottle. Place Target - The front of the rack (specific location); final pose con- strained by rack structure (must be placed on it). 20grab the wine bottle by the neck and place it at the back of the rack put the wine bottle on the rack1.Robot Action-level Constraint: Contact Timing - Must initiate/break contact (grab/release).Trajectory Shape - Requires transfer motion from source (neck) to target (rack back). Ordinal Relations - Strict se- quence: grab before place. 2. Object- level Targeting: Grasp Target - The neck (specific part) of wine bottle. Place Target - The back of the rack (specific location); final pose con- strained by rack structure (must be placed on it). 21grab the wine bottle by the body and place it at the back of the rack put the wine bottle on the rack1.Robot Action-level Constraint: Contact Timing - Must initiate/break contact (grab/release).Trajectory Shape - Requires transfer motion from source (body) to target (rack back). Ordinal Relations - Strict se- quence: grab before place. 2. Object- level Targeting: Grasp Target - The body (specific part) of wine bottle. Place Target - The back of the rack (specific location); final pose con- strained by rack structure (must be placed on it). Continued on next page 19 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 22grab the right edge of this bowl and move it along a diagonal path to the center of the stove put the bowl on the stoveRobot Action-level Constraint: Tra- jectory shape - Diagonal path; Con- tact timing - Grab (initial contact) Object-level Targeting: Target part - Right edge; Object - Bowl; Final pose - Center of the stove 23grab the left edge of this bowl and move it along a diagonal path to the center of the stove put the bowl on the stove Robot Action-level Constraint: Tra- jectory shape - Diagonal path; Con- tact timing - Grab (initial contact). Object-level Targeting: Target part - Left edge; Object - Bowl; Final pose - Center of the stove 24hold the wine bottle firmly by the neck and position it on the upper sec- tion of the cabinet top, orienting the bottle so that its label is facing to the left put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on upper section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle: Label facing left. 25hold the wine bottle firmly by the neck and position it on the bottom section of the cabinet top, orienting the bottle so that its label is facing to the right put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on lower section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle:Label facing right. 26hold the wine bottle firmly by the neck and position it on the upper sec- tion of the cabinet top, orienting the bottle so that its label is facing to the left put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on upper section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle: Label facing left. 27hold the wine bottle firmly by the neck and position it on the bottom section of the cabinet top, orienting the bottle so that its label is facing to the left put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on lower section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle: Label facing left. Continued on next page 20 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 28hold the wine bottle firmly by the neck and position it on the upper sec- tion of the cabinet top, orienting the bottle so that its label is facing to the front put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on upper section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle:Label facing front. 29hold the wine bottle firmly by the neck and position it on the bottom section of the cabinet top, orienting the bottle so that its label is facing to the front put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on lower section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle:Label facing front. 30hold the wine bottle firmly by the neck and position it on the upper sec- tion of the cabinet top, orienting the bottle so that its label is facing to the back put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on upper section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle:Label facing back. 31hold the wine bottle firmly by the neck and position it on the bottom section of the cabinet top, orienting the bottle so that its label is facing to the back put the wine bottle on top of the cabi- net 1.Robot Action-level Constraint: State/Trajectory: Hold firmly, po- sition on lower section.Contact Timing:Grip neck before posi- tioning. Ordinal Relations: Orient after positioning. 2. Object-level Targeting:Handle/Edge/Surface: Neck (grip), cabinet top (placement). Pose/Facing/Angle:Label facing back. 21 KineVLA: Towards Kinematics-Aware Vision-Language-Action Models with Bi-Level Action Decomposition Table 7. Kine-Realman Dataset No.Kinematic language instructionCoarse-grained CoTFine-grained CoT 1pick up the bread and move it to the left front move the breadRobot Action: Trajectory Shape - To the left front; Ordinal Relations - Pick first, then move. Object Targeting: Target Object - The bread; Spatial Relation - Move it to a location (left front). 2 pick up the bread and move it to the right front move the bread Robot Action: Trajectory Shape - To the right front; Ordinal Relations - Pick first, then move. Object Targeting: Target Object - The bread; Spatial Relation - Move it to a location (right front). 3 pick up the bread and move it to the front move the bread Robot Action: Trajectory Shape - To the front; Ordinal Relations - Pick first, then move. Object Targeting: Target Object - The bread; Spatial Relation - Move it to a location (front). 4hold the carrot by the upper part and place it on the right side of the plate place the carrot in the plate RobotAction-levelConstraint: State/Trajectory - Move carrot from held to placement pose.Ordinal Relations - Grasp first, then place. Object-level Targeting: Carrot - Target handle (upper part).Plate - Target surface (top side). Spatial Relation - Final pose on the right side of the plate. 5hold the carrot by the upper part and place it on the left side of the plate place the carrot in the plate RobotAction-levelConstraint: State/Trajectory - Move carrot from held to placement pose.Ordinal Relations - Grasp first, then place. Object-level Targeting: Carrot - Target handle (upper part).Plate - Target surface (top side). Spatial Relation - Final pose on the left side of the plate. 6hold the carrot by the lower part and place it on the left side of the plate place the carrot in the plateRobotAction-levelConstraint: State/Trajectory - Move carrot from held to placement pose.Ordinal Relations - Grasp first, then place. Object-level Targeting: Carrot - Target handle (lower part).Plate - Target surface (top side). Spatial Relation - Final pose on the left side of the plate. 7 hold the carrot by the lower part and place it on the right side of the plate place the carrot in the plateRobotAction-levelConstraint: State/Trajectory - Move carrot from held to placement pose.Ordinal Relations - Grasp first, then place. Object-level Targeting: Carrot - Target handle (lower part).Plate - Target surface (top side). Spatial Relation - Final pose on the right side of the plate. 22

Instant research discovery