Paper deep dive
PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models
Hisayuki Yokomizo, Taiki Miyanishi, Yan Gang, Shuhei Kurita, Nakamasa Inoue, Yusuke Iwasawa
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/22/2026, 5:29:43 AM
Summary
PhysQuantAgent is an inference pipeline designed to improve mass estimation for Vision-Language Models (VLMs) in robotic manipulation. It addresses the limitations of current VLMs in physical reasoning by introducing three visual prompting methods—object detection, scale estimation, and cross-sectional image generation—to provide explicit spatial and structural cues. The authors also introduce VisPhysQuant, a new benchmark dataset consisting of RGB-D videos of small, everyday objects with ground-truth mass annotations, demonstrating that visual prompting significantly enhances mass estimation accuracy compared to traditional reconstruction-based methods.
Entities (5)
Relation Signals (4)
PhysQuantAgent → evaluatedon → VisPhysQuant
confidence 100% · we further introduce VisPhysQuant, a dataset... to evaluate the effectiveness of PhysQuantAgent
PhysQuantAgent → utilizes → Vision-Language Models
confidence 100% · PhysQuantAgent, a framework for real-world object mass estimation using VLMs
VisPhysQuant → contains → RGB-D videos
confidence 95% · VisPhysQuant consists of RGB-D videos of real objects
PhysQuantAgent → improves → mass estimation accuracy
confidence 95% · visual prompting significantly improves mass estimation accuracy on real-world data
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-Language Models (VLMs) are increasingly applied to robotic perception and manipulation, yet their ability to infer physical properties required for manipulation remains limited. In particular, estimating the mass of real-world objects is essential for determining appropriate grasp force and ensuring safe interaction. However, current VLMs lack reliable mass reasoning capabilities, and most existing benchmarks do not explicitly evaluate physical quantity estimation under realistic sensing conditions. In this work, we propose PhysQuantAgent, a framework for real-world object mass estimation using VLMs, together with VisPhysQuant, a new benchmark dataset for evaluation. VisPhysQuant consists of RGB-D videos of real objects captured from multiple viewpoints, annotated with precise mass measurements. To improve estimation accuracy, we introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object. Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.
Tags
Links
- Source: https://arxiv.org/abs/2603.16958v1
- Canonical: https://arxiv.org/abs/2603.16958v1
Trouble viewing inline? Open PDF directly →
Full Text
39,172 characters extracted from source content.
Expand or collapse full text
PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models Hisayuki Yokomizo 1 Taiki Miyanishi 1 Yan Gang 1 Shuhei Kurita 2,3 Nakamasa Inoue 2 Yusuke Iwasawa 1 1 The University of Tokyo 2 Institute of Science Tokyo 3 National Institute of Informatics Abstract— Vision-Language Models (VLMs) are increasingly applied to robotic perception and manipulation, yet their ability to infer physical properties required for manipulation remains limited. In particular, estimating the mass of real-world objects is essential for determining appropriate grasp force and ensur- ing safe interaction. However, current VLMs lack reliable mass reasoning capabilities, and most existing benchmarks do not explicitly evaluate physical quantity estimation under realistic sensing conditions. In this work, we propose PhysQuantAgent, a framework for real-world object mass estimation using VLMs, together with VisPhysQuant, a new benchmark dataset for evaluation. VisPhysQuant consists of RGB-D videos of real objects captured from multiple viewpoints, annotated with precise mass measurements. To improve estimation accuracy, we introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross- sectional image generation to help the model comprehend the size and internal structure of the target object. Experiments show that visual prompting significantly improves mass esti- mation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference. I. INTRODUCTION Vision–Language Models (VLMs) jointly process visual inputs and natural language instructions. When integrated into robotic systems, they enable end-to-end perception- to-action pipelines [16, 20, 39]. However, safe physical interaction requires not only semantic understanding but also quantitative reasoning about object properties. In robotic manipulation, selecting an appropriate grip- ping force is critical for safe and stable operation. The required gripping force depends on the object’s mass, size, and material properties [4]. Although force sensors provide feedback after contact, the initial gripping force must be set before interaction. If this initial force is too small, the object may slip or be dropped. If it is too large, the object may deform or be damaged. Accurate vision-based mass estimation before contact is therefore essential for reliable manipulation. Recent approaches such as NeRF2Physics [37] and PUGS [28] estimate object mass by coupling volumetric reconstruction from 3D geometry with material inference using Large-Language Models (LLMs) or VLMs. These methods reconstruct object geometry to compute volume and then infer material properties to approximate density. However, these methods rely on computationally intensive 4 VLM VLM-based Integrated Inference ToolUse + Visual Prompting image image VLM Material Estimation 3D Reconstruction Volume Estimation Previous Mass Inference Robot Grasping Previous Method Ours Failure Success Adjust the Grip Force Mass: 350 g Mass: 120 g PhysQuantAgent (Ours) Grip Force Control Fig. 1. Comparison of previous mass estimation methods and our proposed PhysQuantAgent. While prior approaches rely on computationally expensive 3D reconstruction from RGB-D images, PhysQuantAgent directly infers object mass with a VLM, enabling fast estimation for grasp force control. 3D reconstruction and multi-stage processing pipelines. Such requirements increase computational cost and system com- plexity, which limits the efficiency and scalability in practical robotic applications. An alternative approach is to directly leverage the prior knowledge and spatial reasoning capability of VLMs to estimate mass from RGB-D observations. De- spite their strong reasoning capabilities, it remains unclear whether VLMs can reliably estimate object mass directly from visual observations. Furthermore, most existing object datasets lack mass annotations [1, 26, 31]. Even when mass labels are provided, they are often restricted to large, furniture-scale objects in existing datasets [11], rather than small objects encountered in robotic manipulation. As a result, datasets for mass estimation of small, manipulation- relevant objects remain limited. To address these challenges, we propose PhysQuantA- gent, a plug-and-play inference pipeline that leverages visual prompting [30] to enable efficient and reliable object mass estimation with VLMs. Figure 1 compares previous mass estimation approaches with our proposed PhysQuantAgent. Existing methods typically estimate object mass from RGB- D images through computationally expensive 3D reconstruc- tion pipelines. In contrast, PhysQuantAgent directly infers object mass using a VLM without requiring explicit 3D reconstruction, enabling fast and practical mass estimation. The estimated mass can then be used to adjust the grasp force arXiv:2603.16958v1 [cs.CV] 17 Mar 2026 during robotic manipulation. By explicitly providing scale and structural cues through visual prompts, PhysQuantAgent helps VLMs better infer underlying 3D structure from 2D observations, which improves the estimation of physical quantities such as mass. To evaluate the effectiveness of PhysQuantAgent for vi- sual mass estimation, we further introduce VisPhysQuant, a dataset of 360-degree RGB-D videos of small, robot- graspable everyday objects annotated with precise mass measurements. The dataset is designed to support systematic evaluation of vision-based mass estimation under realistic robotic manipulation settings. Unlike existing object datasets that primarily focus on large objects or lack physical anno- tations, VisPhysQuant provides ground-truth mass labels for objects commonly encountered in robotic grasping tasks. Extensive experiments on VisPhysQuant systematically evaluate the mass inference capability of modern VLMs and show that visual prompting significantly improves estimation accuracy. Furthermore, the results demonstrate that recent VLMs can outperform prior reconstruction-based approaches in visual mass estimation. We further validate the practical effectiveness of PhysQuantAgent through real-world robotic manipulation tasks, comparing it with prior mass-estimation methods. I. RELATED WORK A. Physical Properties Estimation Existing work on physical quantity estimation with VLMs has primarily focused on estimating object length. For ex- ample, SpatialVLM [7] constructs a pipeline that generates VQA-style training data from internet images to improve the quantitative reasoning ability of VLMs. SpatialRGPT [9] adds a module dedicated to depth images and trains them together with RGB images. SpatialBot [5] incorporates depth information as textual inputs to VLMs, whereas SD-VLM [8] encodes depth information directly during training. Several benchmarks have been proposed to evaluate the spatial reasoning ability of VLMs [17, 36]. Beyond geometric quantities such as length, recent studies have also explored estimating physical properties such as reflectance, hardness, and surface roughness from tactile information [25, 35]. As for mass estimation, studies have explored estimating object mass from images using deep neural networks [2, 19, 24]. Recent approaches combine language models with 3D reconstruction techniques, such as NeRF or Gaussian Splatting, to estimate object volume and then infer material properties to approximate mass [28, 37]. However, these methods do not directly estimate the mass using language models and often suffer from large errors, with relative errors reported to be around 80–100%. B. Physical Understanding Benchmarks Planning robotic manipulation with VLMs requires quan- titative understanding of physical properties, such as object mass and material characteristics, in addition to reasoning about object interactions. Existing benchmarks for physical understanding can be broadly categorized into two direc- tions. First, datasets such as CLEVRER [34] and Super- CLEVER [21] evaluate the ability to reason about object interactions, for example, predicting future motion after collisions. Second, benchmarks including Physion++ [29], ContPhy [38], and PhysBench [10] incorporate tasks involv- ing physical quantities. However, these quantity-related tasks are primarily formulated as relative comparisons, rather than requiring estimation of absolute physical values. While such relative reasoning evaluates qualitative physical understand- ing, it does not require estimating absolute physical values. In contrast, robotic manipulation demands the estimation of absolute quantities, e.g., object mass, to determine appro- priate grasp forces and ensure safe interaction. This distinc- tion highlights a fundamental gap between existing physical reasoning benchmarks and the quantitative requirements of real-world robotic manipulation. C. Object Datasets Various object datasets have been proposed as training or evaluation data for tasks such as 3D reconstruction or camera pose estimation. However, they present significant limitations when applied to the mass estimation task. Table I summarizes representative object datasets and highlights their limitations for mass estimation, including missing mass annotations, limited viewpoint coverage, or reliance on synthetic 3D models. For example, although Objaverse [15] is a large- scale object collection, its format is primarily restricted to 3D models. Objectron [1] provides real-world captures, yet its videos cover limited viewpoints, making it difficult to observe the full object structure. Subsequent works have improved visual coverage through 360-degree captures [26, 31] or depth integration [18, 32]. However, these datasets do not provide ground-truth mass annotations. Furthermore, even when scale information is introduced, such as in PhysXNet [6], it is often estimated by models rather than measured physical quantities. Amazon Berkeley Objects (ABO) [11] provides object size and mass annotations and has been used in prior mass estimation studies [28, 37]. However, the dataset primarily consists of product models and rendered images rather than real-world RGB-D cap- tures, and its object categories are largely furniture-scale items. As a result, it contains relatively few small objects commonly encountered in robotic manipulation. To address these limitations, we introduce VisPhysQuant, a dataset that pairs real-world RGB-D video captures with 360-degree object coverage and ground-truth mass annotations, enabling systematic evaluation of visual mass estimation in robotic manipulation scenarios. I. METHOD A. Task Formulation We address vision-based object mass estimation for robotic manipulation with VLMs. Given N multi-view RGB- D observations of a real-world object, I 0 =(I n , D n ) N n=1 , where I n and D n denote the RGB image and corresponding depth map captured from viewpoint n, our goal is to estimate TABLE I COMPARISON OF EXISTING DATASETS. DatasetVideoViewCategoryRealScaleMass OmniObject3D [31]RGBFull190✓✗ CO3D [26]RGBFull50✓✗ WildRGB-D [32]RGBDFull44✓✗ Objectron [1]RGBLimited9✓✗ HANDAL [18]RGBDLimited17✓✗ ABO [11]✗(3D)-63✗✓ VisPhysQuantRGBDFull87✓ the object’s physical mass m ∈R >0 . A VLM predicts the mass as ˆm = V LM(Q,I 0 ),(1) where Q is a textual prompt and V LM(·) denotes the VLM inference function. We define the objective as minimizing the absolute mass error: L =|m− ˆm|. Since the task targets robot-arm manipulation, we assume the robot is equipped with a calibrated RGB-D sensor whose intrinsic parameters (e.g., focal length and sensor size) are known. Together with metric depth measurements, this enables physically grounded scale reasoning for mass estimation. B. PhysQuantAgent Overview. Mass estimation from raw RGB-D observations is fundamentally underconstrained. Although depth provides geometric information, it does not directly reveal physical cues such as volume, density, or calibrated scale. As a result, VLMs often rely on superficial visual cues, leading to unstable mass predictions. To address this limitation, we propose PhysQuantAgent, an inference framework that leverages Visual Prompting [30] to make physically relevant cues explicit to the VLM (see Figure 2). Rather than re- lying solely on unprocessed observations, PhysQuantAgent generates structured visual augmentations that expose com- plementary attributes required for mass estimation. PhysQuantAgent comprises three Visual Prompting mod- ules. Each module is designed to emphasize a distinct physical aspect of the object that is informative for mass esti- mation. Given an RGB-D observation, the VLM analyzes the scene and selects the most appropriate module for the target object. The selected module produces an augmented image that provides relevant visual cues. This adaptive cue selection mechanism improves the reliability of mass estimation. In the inference pipeline, the VLM first selects the Visual Prompting, then returns the image with that applied, and the model infers mass using both images. Figure 3 shows the example of input prompt in each stage. Mass Estimation Tools. In order to improve the accuracy of mass inference by explicitly providing information such as the size and internal structure of an object to the VLM, we introduce three complementary Visual Prompting tools [30]: (a) object detection for target localization, (b) scale estima- tion for geometric reasoning, and (c) cross-sectional image generation for structural understanding. The selected tool generates an augmented image that explicitly highlights a specific physical attribute of the object. 5 select image input input VLM Visual Prompting (VP) VLM Stage1: Tool selection Stage2: Mass estimation using tools Original image VP image (Scale Estimation) Object Detection Scale Estimation Cross-sectional Image Original image Mass estimation Fig. 2.PhysQuantAgent framework: First, the VLM selects the tool to use (Stage1), then estimates the mass of the object from the original image and the image with the tool applied (Stage2). Stage1: Tool selection You will be given an image of an object. Your task is to estimate the weight of the object. Choose which tools to use (you may choose multiple) to help estimate the object’s weight. Available tools: - Object Detection: create an annotated image with detected object bounding box. - Scale Estimation: create an annotated image with measured X/Y lengths. - Cross sectional Image: generate three-view cross- sectional images of the object. Stage2 Mass estimation You are a physics and engineering assistant. Estimate the weight of the object from the images. Reason step by step and finally state your answer in kilograms like ’Answer: - kg’. Fig. 3.Example of the instruction prompt in PhysQuantAgent. (a) Object Detection: Mass estimation must be conditioned on the correct target object. Without explicit localization, a VLM may incorporate surrounding objects and produce biased estimates. To prevent this, we localize the target using Grounding DINO [23] and provide the resulting bounding box to the VLM. This visual constraint restricts attention to the object of interest and reduces background interference. (b) Scale Estimation: Mass is directly related to physical size. However, pixel coordinates do not encode metric scale. We therefore overlay geometric axes with metric information to provide explicit scale cues. Unlike 3DAxiesPrompts [22], which requires manual annotation, our method automatically generates scale references. When RGB-D input is given, metric distances are com- puted from camera intrinsics and depth under the pinhole 11 Aperture O d f h’ Image sensor Real world h h = h’ × d/f Fig. 4.Pinhole camera model. Visual Prompting Cross-sectional Image Scale Estimation Object Detection Input Images Fig. 5.Examples of Visual Prompting. model (see Figure 4). After segmenting the object, we estimate its horizontal and vertical extents and overlay the corresponding metric lengths as scale annotations (see Fig- ure 5). (c) Cross-sectional Image: External appearance alone does not determine mass, as objects with similar shapes may differ in internal material distribution and effective density. To expose cues about internal occupancy, we generate cross- sectional visualizations using an image editing model (e.g., Nano Banana [14]). Conditioned on the input image and text instructions, the model synthesizes sliced views from multiple directions, which are appended to the original observation to support density-aware mass reasoning. IV. DATASET VisPhysQuant is a RGB-D object dataset for evaluating VLMs’ mass estimation ability. Our dataset consists of approximately 300 samples, each annotated with ground- truth object mass as shown in Figure 6. Unlike other object datasets [6, 11], VisPhysQuant provides real-world RGB-D videos with ground-truth mass annotations. This makes it suitable for mass estimation in robotic manipulation scenar- ios. A. Data Collection We collected RGB-D videos using Record3D on a LiDAR- equipped iPhone 16 pro. The application provides synchro- nized RGB frames and metric depth maps reconstructed from onboard LiDAR measurements. This setup enables direct acquisition of geometrically consistent RGB-D observations. Each object was recorded by moving the camera around it to obtain near 360-degree coverage. We maintained a moderate and consistent camera-to-object distance to ensure stable depth reconstruction. Data were discarded if (i) sub- stantial regions of the object were not reconstructed, (i) Category: Calculator Mass: 0.2 kg Category: Spoon Mass: 0.011 kg Category: Cup Mass: 0.003 kg Category: Glue gun Mass: 0.242 kg Category: Pen Mass: 0.009 kg Category: Robot arm Mass: 0.7 kg Category: Wipe Mass: 0.47 kg Category: Plate Mass: 0.011 kg Fig. 6. VisPhysQuant contains approximately 300 RGBD videos of objects with mass annotations. To ensure diversity, the data were collected in diverse background settings. severe camera shake degraded depth quality, or (i) the object was partially outside the field of view. These criteria ensure reliable geometric information for subsequent scale reasoning. The ground-truth mass of each object was measured using a calibrated digital scale, TANITA KJ-212 (precision: ±0.3 g). For objects that could not be weighed directly due to size or safety constraints, we used manufacturer-provided specifications. In such cases, we verified consistency between catalog values and approximate manual measurements when possible. Objects were placed individually or in moderately clut- tered tabletop environments to reflect realistic household ma- nipulation scenarios. The backgrounds consisted of common indoor surfaces such as wood, plastic, and fabric under typi- cal indoor lighting conditions. The camera-to-object distance ranged from approximately 0.3 to 0.5 meters to ensure stable RGB-D capture. Both isolated-object and cluttered-context recordings are included to evaluate robustness to contextual interference. B. Dataset Statistics We analyze the category distribution of VisPhysQuant and compare it with ABO [11], a widely used object dataset that has been adopted in prior mass estimation studies [28, 37]. Figure 7 shows the category distributions of the two datasets. ABO is furniture-centric and exhibits limited object diversity, whereas VisPhysQuant contains a broader range of objects that can be grasped by humans or robotic manipulators. Figure 8 compares the mass distributions of the two datasets. ABO primarily contains furniture-scale objects with a median mass of 12.7 kg. In contrast, VisPhysQuant focuses on small- and medium-sized objects that are suitable for robotic manipulation and enables systematic evaluation of mass estimation within realistic manipulation ranges. The mass range of VisPhysQuant spans from approximately 0.001 kg to 5 kg, with a median of 0.08 kg. This range aligns with the payload capacity of common household and research robotic manipulators, reflecting real- Fig. 7.A comparison of category distribution of ABO [11] (left) and VisPhysQuant (right) Fig. 8.A comparison of weight distribution of ABO [11] and Vis- PhysQuant. istic manipulation tasks such as grasping and pick-and-place. Notably, VisPhysQuant also includes objects below 100 g, which are underrepresented in existing large-scale 3D object datasets but frequently appear in everyday manipulation scenarios. V. EXPERIMENT A. Experimental Setup Benchmarks and Metrics. We evaluate VLMs on the mass estimation task using VisPhysQuant. We use Minimum Ratio Error (MnRE) as a metric, following prior work [28, 37]. MnRE is defined as MnRE = min m ˆm , ˆm m ,(2) where m denotes the ground-truth mass and ˆm the pre- dicted mass. MnRE provides a symmetric and scale-invariant measure that penalizes underestimation and overestimation equally. Compared with absolute or relative error, MnRE avoids bias toward overestimation or underestimation and provides a more balanced evaluation of mass prediction accuracy. Baselines.WecomparePhysQuantAgentwith NeRF2Physics [37], a reconstruction-based approach for mass estimation. We also evaluate VLM baselines without visual prompting, including Qwen3-VL-8B, Gemini 2.5 pro and Gemini 3.1 pro [3, 12, 13]. Although we also considered PUGS [28], preliminary experiments with the publicly available implementation on the ABO dataset showed lower performance than NeRF2Physics, and we Fig. 9.The mass estimation performance of NeRF2Physics and VLMs with PhysQuantAgent. therefore adopt NeRF2Physics as the reconstruction-based baseline. Implementation Details. To process videos with VLMs, we extract sequential frames. Specifically, we sample frames every 30 frames from the 15-second, 30-fps videos, resulting in approximately 15 images per video. The image generation model used was Gemini 3 Pro Image-preview (Nano Banana Pro [14]). For object detection and length estimation, we use Grounded-Segment-Anything [27]. B. Experimental Results Overall Performance. Figure 9 presents the mass estimation results of VLMs and NeRF2Physics on VisPhysQuant using MnRE as the evaluation metric. Recent VLMs outperform the reconstruction-based method NeRF2Physics even without explicit volume estimation. Applying PhysQuantAgent con- sistently improves the MnRE of each tested VLM, demon- strating the effectiveness of the proposed visual prompting. Ablation Study. Table I presents the ablation study evalu- ating the contribution of each visual prompting module. All prompting strategies improve the MnRE compared with the baseline without visual prompting, indicating that providing explicit spatial or structural cues helps VLMs produce more reliable mass estimates. Figure 10 shows the effect of varying the number of input frames. The result shows that accuracy does not in- crease monotonically with more frames. Each of the tested VLMs achieve the best performance with approximately 5– 10 frames. Using too few frames provides insufficient struc- tural information, whereas too many frames introduce redun- dant observations that slightly degrade inference accuracy. Notably, PhysQuantAgent achieves competitive performance with only a few frames, while reconstruction-based methods such as NeRF2Physics typically require dozens of images (around 30) for volume estimation. Qualitative Results. Figure 11 shows an example where visual prompting significantly improves mass estimation accuracy. By using scale estimation, the VLM can infer an object’s size and volume, which leads to more accurate TABLE I ABLATION OF VISUAL PROMPTS. Base ModelVisual PromptMnRE↑ Gemini-2.5-pro Baseline0.721 + Object Detection0.747 + Scale Estimation0.754 + Cross-sectional Image0.738 Gemini-3.1-pro Baseline0.778 + Object Detection0.806 + Scale Estimation0.789 + Cross-sectional Image0.796 Fig. 10.Ablation on input frame count. mass predictions. On the other hand, visual prompting can also introduce errors, as shown in Figure 12. Distance is estimated using depth measurements obtained from a LiDAR sensor based on the time-of-flight of emitted laser pulses. However, LiDAR signals can penetrate transparent objects such as glass, resulting in overestimated depth values. In such cases, depth estimation models such as Depth Anything [33] could help mitigate this issue. Cross-sectional image gen- eration sometimes also fail to produce appropriate structural references. The generated images sometimes contain artifacts or non-existent objects, which can lead to overestimation of object mass. C. Robot Application We evaluate the practical utility of PhysQuantAgent in a robotic manipulation task using the xArm7. Figure 13 demonstrates this robotic manipulation scenario. First, we record a video of the target object and estimate its mass using mass estimation methods. Based on the estimated mass, the gripping force of the robot arm is adjusted by modulating the applied current. The gripper width is automatically determined by gradually closing the gripper until contact with the object is detected. As shown in Figure 1, inaccurate mass estimation from NeRF2Physics leads to insufficient gripping force, causing the robot to fail to lift the object. In contrast, PhysQuantAgent provides more reliable mass estimates, enabling stable robotic grasping. Output A rough estimation (volume × density) A standard length for such kitchen tongs is around 25-30 cm Volume Estimation Handles: Two arms, roughly 20 cm long, 1.5 cm wide, and 0.8 cm thick. Volume ≈ 2 * (20 cm * 1.5 cm * 0.8 cm) = 48 cm³. Heads: Two heads, each about 8 cm long and 5 cm wide. Approximating their average thickness and shape, their combined volume is likely around 30-40 cm³. The slots in one head reduce its volume slightly. Total Volume: Adding the parts together, a rough estimate for the total volume is approximately 90-110 cm³. Let's use 100 cm³ as a working estimate. Density Estimation The density of nylon is about 1.14 g/cm³, and silicone rubber is around 1.1-1.5 g/cm³. A reasonable average density for this type of kitchen utensil is about 1.2 g/cm³. Answer: 100 cm³ × 1.2 g/cm³ = 0.12kg Output Inference based on accurate length From the provided scale estimation, the dimensions of the tongs are: Length: 16.87 cm, Width: 7.75 cm The thickness is not directly measured, but visually, it appears to be around 5-8 m on average. Volume Estimation Arms: Let's approximate each arm as a rectangular prism. ・ Length: ~12 cm, Width: ~1.5 cm, Thickness: ~0.5 cm ・ Volume of two arms = 2 × (12 cm × 1.5 cm × 0.5 cm) = 18 cm³ Heads: Let's approximate the two heads (one solid, one slotted). ・ Length: ~5 cm, Width: ~4 cm, Thickness: ~0.6 cm ・ Volume of the solid head = 5 cm × 4 cm × 0.6 cm = 12 cm³ ・ The slotted head has material removed. We'l estimate its volume as 75% of the solid head's volume: 12 cm³ × 0.75 = 9 cm³ Connecting Bend: We'l add a small amount for the U-shaped bend, approximately 5 cm³. Total Estimated Volume: 18 cm³ + 12 cm³ + 9 cm³ + 5 cm³ ≈ 44 cm³. To account for the fillets and slightly thicker parts, we can round this estimate up to approximately 50 cm³. Density Estimation The density of common materials for such tongs is: Nylon: ~1.14 g/cm³, Silicone: ~1.1 - 1.5 g/cm³ A reasonable average density to assume for this type of kitchen utensil is 1.2 g/cm³. Answer: 0.06kg Base Base + PhysQuantAgent Question: You are a physics and engineering assistant. Estimate the weight of the object from the images. Reason step by step and finally state your answer in kilograms like 'Answer:〇kg'. GT: 0.05kg Fig. 11.Qualitative example of PhysQuantAgent. Fig. 12. Error cases in Visual Prompting. Scale estimation can yield large errors for transparent objects (left), while Cross-sectional Image generation is prone to hallucinations, such as the synthesis of non-existent objects (right). VI. CONCLUSION We propose PhysQuantAgent, a plug-and-play inference pipeline for object mass estimation that integrates tool- based visual prompting with VLM reasoning. Our results demonstrate that explicitly providing spatial cues enables VLMs to perform more reliable physical inference. We also introduced VisPhysQuant, a dataset of real-world RGB-D videos of small, robot-graspable everyday objects annotated with ground-truth mass measurements, which provides a benchmark for evaluating visual mass estimation. The pro- posed method and dataset enable more reliable estimation of object mass from visual observations, facilitating grasp-force adjustment in robotic manipulation. REFERENCES [1]Adel Ahmadyan, Liangkai Zhang, Artsiom Ablavatski, Jianing Wei, and Matthias Grundmann. “Objectron: A large scale dataset of object-centric videos in the wild with pose annotations”. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021, p. 7822–7831. Fig. 13.Example of a robotic manipulation task. [2]Jo ̃ ao Martinho Lopes Andrade and Plinio Moreno. “Improving the Estimation of Object mass from images”. In: 2023 IEEE Interna- tional Conference on Autonomous Robot Systems and Competitions. 2023, p. 199–206. DOI: 10 . 1109 / ICARSC58346 . 2023 . 10129573. [3]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. “Qwen3- vl technical report”. In: arXiv preprint arXiv:2511.21631 (2025). [4]A. Bicchi and V. Kumar. “Robotic grasping and contact: a re- view”. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings. Vol. 1. 2000, 348–353 vol.1. DOI: 10 . 1109 / ROBOT.2000.844081. [5]Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. “Spatialbot: Precise spatial understanding with vision language models”. In: 2025 IEEE International Conference on Robotics and Automation. IEEE. 2025, p. 9490–9498. [6]Ziang Cao, Zhaoxi Chen, Liang Pan, and Ziwei Liu. “Physx-3d: Physical-grounded 3d asset generation”. In: Advances in Neural Information Processing Systems. 2025. [7]Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. “Spatialvlm: Endowing vision- language models with spatial reasoning capabilities”. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, p. 14455–14465. [8]Pingyi Chen, Yujing Lou, Shen Cao, Jinhui Guo, Lubin Fan, Yue Wu, Lin Yang, Lizhuang Ma, and Jieping Ye. “SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision- Language Models”. In: Advances in Neural Information Processing Systems. 2025. [9]An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Rui- han Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. “Spatial- rgpt: Grounded spatial reasoning in vision-language models”. In: Advances in Neural Information Processing Systems 37 (2024), p. 135062–135093. [10]Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. “PhysBench: Benchmarking and Enhancing Vision- Language Models for Physical World Understanding”. In: Interna- tion Conference on Learning Representations. 2025. [11]Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. “Abo: Dataset and benchmarks for real-world 3d object understanding”. In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. 2022, p. 21126–21136. [12]Google DeepMind. Gemini 2.5: Our most intelligent AI model. https://blog.google/innovation-and-ai/models- and- research/google- deepmind/gemini- model- thinking- updates- march- 2025/ [Accessed: 2026-3-2]. 2026. [13]Google DeepMind. Gemini 3.1 Pro: A smarter model for your most complex tasks. https://blog.google/innovation-and- ai/models-and-research/gemini-models/gemini- 3-1-pro/ [Accessed: 2026-3-2]. 2026. [14]Google DeepMind. Introducing Nano Banana Pro. https:// blog.google/innovation-and-ai/products/nano- banana-pro/ [Accessed: 2026-1-15]. 2025. [15]Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. “Objaverse: A universe of annotated 3d objects”. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023, p. 13142–13153. [16]Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Cheb- otar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. “PaLM-E: an embodied multimodal language model”. In: Proceedings of the 40th Interna- tional Conference on Machine Learning. 2023. [17]Mohsen Gholami, Ahmad Rezaei, Zhou Weimin, Sitong Mao, Shunbo Zhou, Yong Zhang, and Mohammad Akbari. Spatial Rea- soning with Vision-Language Models in Ego-Centric Multi-View Scenes. 2025. arXiv: 2509. 06266 [cs.CV]. URL: https : //arxiv.org/abs/2509.06266. [18]Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeffrey Smith, and Stan Birchfield. “Handal: A dataset of real-world manipulable object categories with pose an- notations, affordances, and reconstructions”. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2023, p. 11428–11435. [19]Muhammad Hamdan, Diane Rover, Matthew Darr, and John Just. “Mass estimation from images using deep neural network and sparse ground truth”. In: 2019 18th IEEE International Conference On Machine Learning And Applications. IEEE. 2019, p. 1987–1992. [20]Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. “VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models”. In: 7th Annual Con- ference on Robot Learning. 2023. URL: https://openreview. net/forum?id=9_8LF30mOC. [21]Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Ko- rtylewski, Wufei Ma, Benjamin Van Durme, and Alan L Yuille. “Super-clevr: A virtual benchmark to diagnose domain robustness in visual reasoning”. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023, p. 14963–14973. [22]Dingning Liu, Xiaomeng Dong, Renrui Zhang, Xu Luo, Peng Gao, Xiaoshui Huang, Yongshun Gong, and Zhihui Wang. “3dax- iesprompts: Unleashing the 3d spatial task capabilities of gpt-4v”. In: arXiv preprint arXiv:2312.09738 (2023). [23]Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. “Grounding dino: Marrying dino with grounded pre-training for open-set object detection”. In: European conference on computer vision. Springer. 2024, p. 38–55. [24]Amar Nath, Divya Patel, and Rajdeep Niyogi. “Mass Prediction and Analysis of an Object’s Mass from Its Image Using Deep Learning”. In: SN Computer Science 5.6 (2024), p. 711. [25]Masashi Osada, Gustavo A Garcia Ricardez, Yosuke Suzuki, and Tadahiro Taniguchi. “Reflectance estimation for proximity sensing by vision-language models: Utilizing distributional semantics for low-level cognition in robotics”. In: Advanced Robotics 38.18 (2024), p. 1287–1306. [26]Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. “Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction”. In: Proceedings of the IEEE/CVF international conference on computer vision. 2021, p. 10901–10911. [27]Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. “Grounded sam: Assembling open-world models for diverse visual tasks”. In: arXiv preprint arXiv:2401.14159 (2024). [28]Yinghao Shuai, Ran Yu, Yuantao Chen, Zijian Jiang, Xiaowei Song, Nan Wang, Jv Zheng, Jianzhu Ma, Meng Yang, Zhicheng Wang, et al. “PUGS: Zero-shot Physical Understanding with Gaussian Splatting”. In: International Conference on Robotics and Automa- tion. 2025. [29]Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen, Daniel Bear, Chuang Gan, Josh Tenenbaum, Dan Yamins, Judith Fan, and Kevin Smith. “Physion++: Evaluating physical scene understanding that requires online inference of different physical properties”. In: Advances in Neural Information Processing Systems 36 (2023), p. 67048–67068. [30]Junda Wu, Zhehao Zhang, Yu Xia, Xintong Li, Zhaoyang Xia, Aaron Chang, Tong Yu, Sungchul Kim, Ryan A Rossi, Ruiyi Zhang, et al. “Visual prompting in multimodal large language models: A survey”. In: arXiv preprint arXiv:2409.15310 (2024). [31]Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, et al. “Omniob- ject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, p. 803–814. [32]Hongchi Xia, Yang Fu, Sifei Liu, and Xiaolong Wang. “Rgbd objects in the wild: Scaling real-world 3d object learning from rgb-d videos”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, p. 22378–22389. [33]Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. “Depth anything: Unleashing the power of large-scale unlabeled data”. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024, p. 10371–10381. [34]Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. “Clevrer: Collision events for video representation and reasoning”. In: International Conference on Learning Representations. 2020. [35]Samson Yu, Kelvin Lin, Anxing Xiao, Jiafei Duan, and Harold Soh. “Octopi: Object property reasoning with large tactile-language models”. In: Robotics: Science and Systems. 2024. [36]Songsong Yu, Yuxin Chen, Hao Ju, Lianjie Jia, Fuxi Zhang, Shaofei Huang, Yuhan Wu, Rundi Cui, Binghao Ran, Zaibin Zhang, Zhedong Zheng, Zhipeng Zhang, Yifan Wang, Lin Song, Lijun Wang, Yanwei Li, Ying Shan, and Huchuan Lu. How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective. 2025. arXiv: 2509.18905 [cs.AI]. URL: https://arxiv. org/abs/2509.18905. [37]Albert J Zhai, Yuan Shen, Emily Y Chen, Gloria X Wang, Xinlei Wang, Sheng Wang, Kaiyu Guan, and Shenlong Wang. “NeRF2Physics: Physical property understanding from language- embedded feature fields”. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition. 2024, p. 28296– 28305. [38]Zhicheng Zheng, Xin Yan, Zhenfang Chen, Jingzhou Wang, Qin Zhi Eddie Lim, Joshua B Tenenbaum, and Chuang Gan. “ContPhy: Con- tinuum Physical Concept Learning and Reasoning from Videos”. In: International Conference on Machine Learning. PMLR. 2024. [39]Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. “Rt-2: Vision-language-action models transfer web knowledge to robotic control”. In: Conference on Robot Learning. PMLR. 2023, p. 2165–2183.