Paper deep dive
Iterative Grasp Pose Refinement: A Deep Reinforcement Learning Approach for 2D Vision
Amir Arsalan Nematollahi, Shayan Ahmadi, Mehdi Tale Masouleh, Ahmad Kalhor
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/19/2026, 5:13:46 AM
Summary
This paper presents a Deep Reinforcement Learning (DQN) framework for iteratively refining robotic grasp poses. The system takes initial grasp candidates generated by a geometric algorithm from 2D overhead images and adjusts their parameters (x, y, theta, w) to transform failed grasps into successful ones. Experiments on the Dex-Net dataset using a UR5 manipulator in simulation achieved a 100% success rate on previously ungraspable objects. The framework's effectiveness was further validated through sim-to-real transfer using a Delta parallel robot.
Entities (7)
Relation Signals (6)
Deep Q-Network → usedfor → Grasp Refinement
confidence 98% · This work proposes a reinforcement learning-based framework for robotic grasp refinement, integrating keypoint-based object representations with a Deep Q-Network (DQN).
Delta parallel robot → usedfor → Sim-to-Real Validation
confidence 95% · The framework's sim-to-real transferability is further validated through physical experiments on a Delta parallel robot
CoppeliaSim → usedfor → Simulation
confidence 95% · The CoppeliaSim environment was employed for simulation due to its robust and feature-rich platform capabilities.
Dex-Net → usedin → Simulation Experiments
confidence 95% · Experiments conducted on 300 objects from the Dex-Net dataset using a UR5 manipulator demonstrate the framework's effectiveness
UR5 → usedin → Simulation Experiments
confidence 95% · Experiments conducted on 300 objects from the Dex-Net dataset using a UR5 manipulator demonstrate the framework's effectiveness
Sabzejou et al. → proposed → Geometric-based Algorithm
confidence 90% · a geometric-based algorithm, which was proposed by Sabzejou et al. [22], was utilized to initially label these objects
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Developing robots capable of understanding and manipulating objects requires compact, interpretable, and generalizable representations. This work proposes a reinforcement learning-based framework for robotic grasp refinement, integrating keypoint-based object representations with a Deep Q-Network (DQN). Using 2D overhead images captured in a simulated environment, a geometric-based algorithm generates initial grasp candidates, which are iteratively refined by the proposed framework, transforming failed grasps into successful ones. Experiments conducted on 300 objects from the Dex-Net dataset using a UR5 manipulator demonstrate the framework's effectiveness, achieving a 100% success rate on objects previously deemed ungraspable by geometrical methods. The framework's sim-to-real transferability is further validated through physical experiments on a Delta parallel robot, where a refined grasp successfully manipulates an object that was previously ungraspable. The findings underscore the effectiveness of reinforcement learning in addressing challenges in robotic grasping, offering a scalable and adaptable solution for contact-rich manipulation tasks.
Tags
Links
- Source: https://arxiv.org/abs/2608.17628v1
- Canonical: https://arxiv.org/abs/2608.17628v1
Trouble viewing inline? Open PDF directly →
Full Text
30,204 characters extracted from source content.
Expand or collapse full text
@headings Iterative Grasp Pose Refinement: A Deep Reinforcement Learning Approach for 2D Vision Amir Arsalan Nematollahi Shayan Ahmadi Mehdi Tale Masouleh Ahmad KalhorHuman and Robot Interaction Laboratory, School of Electrical and Computer Engineering,University of Tehran, Tehran, Iran a.nematollahi shayanahmadi m.t.masouleh akalhor@ut.ac.ir Thanks: $ˆ∗$Equal contribution. Abstract Developing robots capable of understanding and manipulating objects requires compact, interpretable, and generalizable representations. This work proposes a reinforcement learning-based framework for robotic grasp refinement, integrating keypoint-based object representations with a Deep Q-Network (DQN). Using 2D overhead images captured in a simulated environment, a geometric-based algorithm generates initial grasp candidates, which are iteratively refined by the proposed framework, transforming failed grasps into successful ones. Experiments conducted on 300 objects from the Dex-Net dataset using a UR5 manipulator demonstrate the framework’s effectiveness, achieving a 100% success rate on objects previously deemed ungraspable by geometrical methods. The framework’s sim-to-real transferability is further validated through physical experiments on a Delta parallel robot, where a refined grasp successfully manipulates an object that was previously ungraspable. The findings underscore the effectiveness of reinforcement learning in addressing challenges in robotic grasping, offering a scalable and adaptable solution for contact-rich manipulation tasks. Index Terms: Robotics, Grasping, Reinforcement Learning, Grasp Refinement, Object Grasping I Introduction Object manipulation remains a fundamental challenge in robotics, with crucial applications in industrial automation and domestic assistance. Despite significant progress, current systems still lack the human-like dexterity and adaptability needed for unstructured environments. To bridge this gap, data-driven approaches like deep Reinforcement Learning (RL) have become powerful tools. By leveraging trial-and-error interaction, deep RL learns robust and adaptable grasping strategies directly from sensory data [1, 2, 3]. Effective object representations are crucial for enhancing these learned behaviors. Keypoint representations, in particular, allow a robot to focus on critical geometric features, facilitating more precise and efficient manipulation. Integrating keypoint detection with RL is a promising path toward creating reliable, versatile grasping systems that can generalize across diverse objects [4, 5, 6]. Data-driven grasping pipelines primarily rely on supervised learning or reinforcement learning [7]. Supervised approaches train on large, pre-labeled datasets of grasp candidates, a method that can be computationally demanding despite yielding strong generalization [8, 9, 10]. RL has proven effective for learning complex, end-to-end manipulation skills through environmental interaction [11, 12, 13, 14, 15, 16]. However, widespread success is often hindered by challenges like sample efficiency and sparse rewards, which has prompted research into demonstration-guided learning [17] and efficient architectures like graph-based RL frameworks [18]. While end-to-end deep learning is a popular data-driven approach for robotic manipulation [5, 19, 20, 6], the resulting ”black-box” representations often lack the interpretability essential for generalizable control. Keypoint representations offer a structured alternative, providing a compact understanding of object geometry that improves generalization in contact-rich tasks [4, 5, 21]. However, the scalability of these methods has been hindered by a reliance on manual 3D annotation [4, 19]. Even recent automated 2D keypoint generation methods depend on traditional geometric techniques for the entire grasping process, which can overlook viable grasp points and restrict adaptability [22]. This highlights a clear need for a more robust refinement strategy. This work introduces a novel framework that bridges the gap between traditional geometric methods and data-driven learning for robotic grasping. The primary contribution of the present work is a Deep Q-Network (DQN) agent designed specifically for the task of refinement. Rather than learning to grasp, the proposed framework takes failed grasp candidates generated by a geometric algorithm from a 2D overhead image and iteratively adjusts their parameters—represented as (x,y,θ,w)(x,y,θ,w)—to optimize grasp stability and success rates. This work offers a distinct approach, complementing end-to-end grasp synthesis as a refinement module. By focusing the RL problem on local optimization around promising candidates rather than on global exploration, this method offers a more tractable, efficient, and scalable solution. The significance of this methodology lies in its demonstrated ability to systematically convert failed or suboptimal grasp attempts into successful, stable configurations. This enhances the overall robustness of automated grasping systems and presents a pragmatic pathway toward more adaptive and reliable robotic manipulation in complex tasks. The model’s real-world performance was validated using a Delta parallel robot, shown in Fig. 1. The remainder of this article is organized as follows. Section I explains the simulation setup, covering the simulation environment configuration, dataset characteristics and the methodology used for the initial labeling of objects through keypoint assignment. Section I outlines the methodology, including the grasp refinement framework, scoring system, state and action definitions, DQN implementation, and the training procedure. Section IV summarizes the findings and discusses them. Finally, conclusions are presented in Section V. Fig. 1: The Delta parallel manipulator, equipped with a two-finger parallel-jaw gripper, is shown over the workspace. An overhead RGB camera captures the workspace. Fig. 2: The workflow of the refinement process. The initial grasp candidate, generated through geometric labeling, is refined within the simulation environment. Using reinforcement learning, the refinement stage iteratively adjusts the grasp pose (x,y,θ,w)(x,y,θ,w) through actions to maximize stability and minimize slippage, transforming a failed grasp into a successful one. The scoring mechanism, serving as the reward, evaluates each state and guides the agent toward optimal grasp configurations. I Dataset Generation and Labeling This section elaborates on the simulation environment, the dataset used, how the simulation was employed to process the dataset, and the methodology of labeling (assigning key points) for objects. The simulation setup is described in detail below: I-A Simulation Environment and Configuration The CoppeliaSim environment was employed for simulation due to its robust and feature-rich platform capabilities. This environment offers precise physics-based modeling and simulation, making it ideal for tasks like robotic grasping. A UR5 manipulator robot equipped with a RG2 gripper was used in the simulation. Additionally, an RGB-D vision sensor integrated into the simulation environment captured images of the virtual workspace from an overhead view, providing a comprehensive perspective for grasp analysis. The CoppeliaSim Python API facilitated real-time data exchange. I-B Object Dataset The dataset used for this project was a random subset of the Dex-net dataset [23], which contains 300 diverse object meshes. These meshes were imported into the CoppeliaSim environment for simulation. In order to facilitate the grasping process, a geometric-based algorithm, which was proposed by Sabzejou et al. [22], was utilized to initially label these objects by evaluating all possible grasp configurations. After the initial labels were obtained, they were tested in the simulator to observe whether they were successful grasps. Notably, 52 objects in the dataset did not exhibit any successful grasp configurations during the initial evaluation. From these, 14 objects were excluded from the refinement study as their geometries exceeded the physical limitations of the RG2 gripper (i.e., the maximum opening width). For the remaining 38 objects, one of the failed grasp candidates was randomly selected and subjected to the proposed refinement algorithm. This approach aimed to determine whether the algorithm could convert failed grasps into successful ones, demonstrating the robustness and adaptability of the proposed refinement method. I-C Initial Labeling To initiate the labeling process, the objects were imported into the CoppeliaSim environment, and an RGB-D camera was used to capture overhead images of the scene. These images served as input for the previously mentioned geometric-based algorithm, proposed by Sabzejou et al. [22], which acted as an initial labeler. Labeling, in this context, refers to identifying initial grasp configurations for each object based on their geometrical shape. After obtaining the initial grasp candidates, they were tested in the simulation environment. The performance of each configuration was evaluated based on the quality of the grasp regarding stability and lift success. Scores were then assigned to each grasp configuration, providing a baseline for further refinement. I Grasp Refinement Framework I-A Overview The overall framework of the proposed system is illustrated in Fig. 2. The primary objective of the refinement method is converting failed grasps into successful ones and also enhancing the quality of already identified successful grasps regarding stability. The process begins with an initial grasp pose, which serves as input to the refinement model. The model systematically modifies the dimensions of the grasp pose and evaluates each variation in the simulation environment. Before the manipulator attempts to grasp an object, the position and orientation of the object are recorded. This initial state, referred to as the “initial stage,” provides a baseline for comparison. After the grasp attempt, the position and orientation of the object are recorded again, referred to as the “final stage.” Four outcomes are possible for each grasp pose: 1) The manipulator fails to lift the object, instead merely pushing it across the surface or leaving it unmoved. 2) The manipulator lifts the object, but due to excessive slippage caused by an improper grasp pose, the object falls off the gripper. 3) The manipulator successfully lifts the object to an appropriate height (fully lift the object from the surface) with some rotation. 4) The manipulator successfully lifts the object to an appropriate height with negligible slippage or instability. After the manipulator attempts to grasp the object using a modified grasp pose, the simulator returns two outputs: a success flag indicating whether the grasp was successful and a score reflecting the stability of the grasp. By iteratively refining the grasp pose, the model aims to identify configurations that achieve the highest scores. I-B Scoring and Reward Structure The scoring system evaluates grasp stability by measuring the amount of rotational slip (tilt), calculated during the grasp. Minimizing slippage is essential for achieving stable and secure grasps. Scoring is calculated based on the differences in the object’s orientation between the initial and final stages. The simulator provides roll, pitch, and yaw values for both stages, which are used to compute the rotational slip (|Δθ|)(| θ|). Given that the maximum possible angular deviation is π radians, the final score was normalized to a [0,1][0,1] range for successful grasps and to a [−1,0][-1,0] range for failure grasps: Score =1−|Δθ|πsuccessful grasp−1+|Δθ|πfailure graspScore = cases1- | θ|π&successful grasp\\ \\ -1+ | θ|π&failure grasp cases (1) A perfectly stable, tilt-free grasp results in a score of +1+1. Grasp failure is defined as the complete closure of the gripper in the final stage, indicating the absence of an object between its two fingers; this typically manifests as the object dropping during the lifting phase or the gripper being unable to lift the object from its initial position. For successful grasps (lift completed), smaller rotation differences between the initial and final stages indicate more stable and precise grasps, resulting in higher scores (approaching +1+1). For failed grasps, the score reflects the extent to which the object was manipulated despite the failure. A completely missed grasp (no contact) yields −1-1, while grasps that make contact and partially move or lift the object (even if it eventually falls) receive higher scores approaching 0. Thus, within the failed range [−1,0)[-1,0), greater object motion or rotation indicates a grasp closer to success. I-C Actions and States Definition The state in this work is defined by the grasp configuration, represented by a rectangle with four continuous parameters: the Cartesian coordinates of the rectangle’s center (x,y)(x,y), its in-plane rotation (θ)(θ), and its width (w)(w). The state space is represented as a 4D vector (x,y,θ,w)(x,y,θ,w), which provides sufficient parameters for a two-finger robotic gripper. Actions are defined as discrete, small adjustments applied to the state parameters. The action space, represented as (Δx,Δy,Δθ,Δw)( x, y, θ, w), consists of incremental changes to the state variables which are selected from a discrete set of predefined small adjustments. These modifications allow the algorithm to explore and refine grasp configurations effectively. Additionally, a no-op action (no operation) is included, which leaves the state unchanged. This action is useful when the network determines that the current state is already optimal. The specific adjustments corresponding to each discrete action are detailed in Table I. TABLE I: Discrete Action Space for Grasp Refinement. Action Index (a) Adjustment (Δx,Δy,Δθ,Δw)( x, y, θ, w) Description 0 (+1,0,0,0)(+1,0,0,0) Adjust X-pixel 1 (−1,0,0,0)(-1,0,0,0) Adjust X-pixel 2 (0,+1,0,0)(0,+1,0,0) Adjust Y-pixel 3 (0,−1,0,0)(0,-1,0,0) Adjust Y-pixel 4 (0,0,+2,0)(0,0,+2,0) Adjust rotation 5 (0,0,−2,0)(0,0,-2,0) Adjust rotation 6 (0,0,0,+2)(0,0,0,+2) Adjust width 7 (0,0,0,−2)(0,0,0,-2) Adjust width 8 (0,0,0,0)(0,0,0,0) No operation I-D DQN Implementation and Training The refinement algorithm is implemented using a Deep Q-Network (DQN) [24], a technique that utilizes deep neural networks to approximate the Q-value function. A value-based DQN was selected as it is a sufficient and computationally efficient approach for the local refinement task, given the problem’s low-dimensional state space. The DQN is trained to approximate the optimal Q-value function, Q(s,a∣ϕ)Q(s,a φ), where ϕφ represents the network parameters. In this framework, the core components are defined as follows: • State (s): The state st=(x,y,θ,w)s_t=(x,y,θ,w) is the 4D grasp configuration vector at time t. • Action (a): The action ata_t is one of the 9 discrete adjustments (Δx,Δy,Δθ,Δw)( x, y, θ, w) selected from the predefined action space (Table I). • Reward (r): The immediate reward rtr_t is the stability score, ranging from [−1,+1][-1,+1], returned by the scoring mechanism. This score is calculated based on the object’s rotational slip after a grasp attempt. The network’s parameters ϕφ are optimized by sampling mini-batches of experiences (s,a,r,s′)(s,a,r,s ) from an experience replay buffer and minimizing the Mean Squared Error (MSE) loss: L(ϕ)=(s,a,r,s′)[(Q(s,a∣ϕ)−y)2],L(φ)=E_(s,a,r,s ) [ (Q(s,a φ)-y )^2 ], (2) where the target y is defined as: y=r+γmaxa′Q(s′,a′∣ϕ),y=r+γ _a Q(s ,a φ), (3) Here, γ is the discount factor, which was set to 0.9. The Q-network is a fully connected neural network with three hidden layers (256, 128, and 64 neurons, respectively), each using a ReLU activation function and Layer Normalization. The network was trained using the Adam optimizer with a learning rate of 0.005. For exploration, an ϵε-greedy policy was employed; the exploration rate ϵε was initially set to 1.0 and decayed by a factor of 0.999 per step to a minimum of 0.01. The agent was trained for E=25E=25 episodes for each object, with each episode consisting of a maximum of T=120T=120 steps. The agent’s experience replay buffer was sampled using a batch size of 32. This process enables the agent to iteratively refine the grasp configuration by applying small displacement actions (Δx,Δy,Δθ,Δw)( x, y, θ, w) to the current pose. The learning process, which refines an initial grasp candidate, is summarized in Algorithm 1. Algorithm 1 Grasp Refinement using Deep Q-Learning Input: Initial failed grasp sinit=(x,y,θ,w)s_init=(x,y,θ,w), action set A (Table I), Number of episodes E, max steps per episode T Output: The refined grasp configuration sbests_best 1: Initialize Q-network QϕQ_φ with random weights 2: Initialize target network Qϕ′Q_φ with weights ϕ′←ϕφ ←φ 3: Initialize replay buffer D 4: Initialize exploration rate ϵε 5: Initialize sbest←sinits_best← s_init and scorebest←−1.0score_best←-1.0 6: for episode=1episode=1 to E do 7: st←sinits_t← s_init 8: for t=1t=1 to T do 9: Select ata_t using ϵε-greedy policy based on Qϕ(st)Q_φ(s_t) 10: Calculate next state st+1s_t+1 by applying ata_t to sts_t 11: Execute grasp st+1s_t+1 in simulator N times 12: rt←AverageScore(st+1)r_t (s_t+1) 13: Store transition (st,at,rt,st+1)(s_t,a_t,r_t,s_t+1) in D 14: if rt>scorebestr_t>score_best then 15: scorebest←rtscore_best← r_t 16: sbest←st+1s_best← s_t+1 17: end if 18: st←st+1s_t← s_t+1 19: if |||D| is large enough then 20: Sample a random minibatch from D 21: Perform a gradient descent step on the loss 22: function (from Eq. (2)) to update ϕφ 23: end if 24: if rt=1r_t=1 then 25: break 26: end if 27: end for 28: Decay exploration rate ϵε 29: Update target network: ϕ′←ϕφ ←φ 30: end for 31: return sbests_best Fig. 3: Outcomes of challenging objects. Fig. 4: Average reward per episode and the corresponding grasp configuration. Fig. 5: Average reward per episode and the corresponding grasp configuration. IV Results and Discussion The proposed framework underwent a rigorous training procedure to evaluate its performance. A random subset of 300 objects from the Dex-Net dataset [23] was used, and 2D top-view images of these objects were labeled using the geometric-based algorithm proposed by Sabzejou et al. [22]. The algorithm processes each object separately, and the resulting labeled grasp configurations were subsequently tested in a simulation environment. Among the 300 objects, 14 of them could not be grasped due to physical limitations, such as the maximum opening width of the gripper, so they were excluded from the analysis. From the remaining 286 objects, 38 objects were found to lack any grasp configuration that allowed the robot to grasp and lift them to an appropriate height. The proposed refinement framework was applied to these 38 objects. For all of these 38 objects, successful refined grasps were identified. Thus, the success rate of the proposed framework was 100% (Fig. 3). To ensure robustness, each grasp configuration was executed three times and deemed successful only if all trials succeeded; otherwise, the state was labeled a failure. This repeated evaluation confirms that the resulting grasp configurations are reliable and stable. The effectiveness of the DQN training process is demonstrated through Fig. 4 and 5, which display the agent’s average reward per episode and illustrate its progressive improvement in grasp configurations. Fig. 6: Sim-to-real validation of the proposed grasp refinement framework using a pair of pliers. (Top) An initial grasp candidate, generated by the geometric algorithm, proves unstable, leading to failure in both the simulation and the real-world execution. (Bottom) After processing the failed candidate, the proposed RL framework generates a refined grasp pose. This new pose, which was identified as successful in simulation, enables a stable lift with a Delta parallel robot, confirming the direct transfer of the refined grasp’s effectiveness from simulation to a real-world task. Fig. 4(a) and Fig. 5(a) show the initial unsuccessful candidates. Initial attempts yielded minimal rewards, reflecting the agent’s lack of knowledge. As training progressed, the average reward obtained by the agent increased steadily, as shown in Fig. 4(b) and Fig. 5(b). At this stage, the grasp configurations began to show minor improvements, albeit with some residual similarities to the initial failed configurations. Fig. 4(c) and 5(c) illustrate the grasp configurations from episodes with the highest average rewards. These configurations represent the refined grasps identified by the proposed framework. The noticeable differences from initial configurations highlight the effectiveness of the DQN algorithm in optimizing grasp poses. These increased rewards correspond to stable grasps meeting the success criteria. The evolution of average rewards in Fig. 4 and 5 underscores the framework’s learning capabilities, affirming the robustness and adaptability of the proposed refinement framework in improving the quality and success rate of grasping tasks. The progressive refinement of grasp configurations demonstrates the potential of reinforcement learning to address challenges in robotic grasping, particularly for objects initially deemed ungraspable by geometric algorithms. Finally, the proposed framework was evaluated on a physical testbed to verify the transferability of refined grasps from simulation to the real world. An object from the dataset, whose grasp configuration had been labeled as a failure in simulation, was selected; a corresponding physical real object was identified and assigned the same grasp label. The initial (failed) grasp was executed using the Delta parallel manipulator and likewise failed in the real experiment. The failed grasp configuration was then processed by the refinement algorithm, which produced a refined grasp that was successful in simulation. This refined grasp was transferred to the physical object and executed with the manipulator, resulting in a successful and stable lift. An example of this sim-to-real validation is presented in Fig. 6. V CONCLUSION This work introduces a novel reinforcement learning-based framework for refining robotic grasp configurations, combining keypoint-based object representations with a Deep Q-Network (DQN). By leveraging a geometric-based labeling algorithm as a baseline, the framework iteratively optimizes grasp poses, transforming initially failed grasps into stable and precise configurations. Simulation results demonstrate the framework’s effectiveness in addressing limitations of conventional labeling methods, with significant improvements in grasp success rates for challenging objects. From the 52 objects initially labeled ungraspable, 38 fell within hardware limits and the proposed refinement framework produced successful refined grasps for all 38 (100% success). These findings underscore the framework’s ability to systematically explore and refine grasp configurations, even for objects that pose significant challenges due to their geometry or orientation. The progressive increase in rewards validates the DQN’s ability to transform low-reward failures into optimized configurations. This process not only converts failed grasps into successful ones but also enhances the stability of existing candidates. Sim-to-real validations were performed: failed simulated grasp was refined by the proposed algorithm, and the refined configuration yielded a successful lift on the corresponding physical object, confirming the practical effectiveness of the refinement method. Future work will focus on enabling the DQN to generalize better to unseen data. By enhancing its adaptability to diverse and unpredictable scenarios, this research contributes toward the development of more scalable and versatile robotic systems capable of robust manipulation in complex, real-world environments. References [1] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018. [2] V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, J. Pineau et al., “An introduction to deep reinforcement learning,” Foundations and Trends® in Machine Learning, vol. 11, no. 3-4, p. 219–354, 2018. [3] H. Sekkat, O. Moutik, L. Ourabah, B. ElKari, Y. Chaibi, and T. Ait Tchakoucht, “Review of reinforcement learning for robotic grasping: Analysis and recommendations,” Statistics, Optimization & Information Computing, vol. 12, no. 2, p. 571–601, 2024. [4] L. Manuelli, W. Gao, P. Florence, and R. Tedrake, “kpam: Keypoint affordances for category-level robotic manipulation,” in The International Symposium of Robotics Research. Springer, 2019, p. 132–157. [5] K. Fang, Y. Zhu, A. Garg, A. Kurenkov, V. Mehta, L. Fei-Fei, and S. Savarese, “Learning task-oriented grasping for tool manipulation from simulated self-supervision,” The International Journal of Robotics Research, vol. 39, no. 2-3, p. 202–216, 2020. [6] H. S. Koppula, R. Gupta, and A. Saxena, “Learning human activities and object affordances from rgb-d videos,” The International journal of robotics research, vol. 32, no. 8, p. 951–970, 2013. [7] K. F. Dukor and T. Afonja, “A survey: Robot grasping,” 2021. [8] V. Satish, J. Mahler, and K. Goldberg, “On-policy dataset synthesis for learning robot grasping policies using fully convolutional deep networks,” IEEE Robotics and Automation Letters, vol. 4, no. 2, p. 1357–1364, 2019. [9] H. Hosseini, M. T. Masouleh, and A. Kalhor, “Improving the successful robotic grasp detection using convolutional neural networks,” in 2020 6th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS). IEEE, 2020, p. 1–6. [10] J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg, “Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics,” arXiv preprint arXiv:1703.09312, 2017. [11] J. Kober, J. A. Bagnell, and J. Peters, “Reinforcement learning in robotics: A survey,” The International Journal of Robotics Research, vol. 32, no. 11, p. 1238–1274, 2013. [12] A. Mandlekar, D. Xu, R. Martín-Martín, S. Savarese, and L. Fei-Fei, “Learning to generalize across long-horizon tasks from human demonstrations,” arXiv preprint arXiv:2003.06085, 2020. [13] T. Wang, H. Zhang, W. Q. Toh, H. Zhu, C. Tan, Y. Wu, Y. Liu, and W. Jing, “Efficient robotic task generalization using deep model fusion reinforcement learning,” in 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2019, p. 148–153. [14] Y. Zhu, Z. Wang, J. Merel, A. Rusu, T. Erez, S. Cabi, S. Tunyasuvunakool, J. Kramár, R. Hadsell, N. de Freitas et al., “Reinforcement and imitation learning for diverse visuomotor skills,” arXiv preprint arXiv:1802.09564, 2018. [15] D. Quillen, E. Jang, O. Nachum, C. Finn, J. Ibarz, and S. Levine, “Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods,” in 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, p. 6284–6291. [16] F. Zhang, J. Leitner, M. Milford, B. Upcroft, and P. Corke, “Towards vision-based deep reinforcement learning for robotic motion control,” arXiv preprint arXiv:1511.03791, 2015. [17] S. James and A. J. Davison, “Q-attention: Enabling efficient learning for vision-based robotic manipulation,” IEEE Robotics and Automation Letters, vol. 7, no. 2, p. 1612–1619, 2022. [18] A. R. Moghadam, M. T. Masouleh, and A. Kalhor, “Grasp the graph (gtg): A super light graph-rl framework for robotic grasping,” in 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM). IEEE, 2023, p. 861–868. [19] R. Holladay, T. Lozano-Pérez, and A. Rodriguez, “Force-and-motion constrained planning for tool use,” in 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2019, p. 7409–7416. [20] C. Lovchik and M. A. Diftler, “The robonaut hand: A dexterous robot hand for space,” in Proceedings 1999 IEEE international conference on robotics and automation (Cat. No. 99CH36288C), vol. 2. IEEE, 1999, p. 907–912. [21] Y. Park, V. Lepetit, and W. Woo, “Multiple 3d object tracking for augmented reality,” in 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 2008, p. 117–120. [22] A. Sabzejou, M. T. Masouleh, and A. Kalhor, “2d skeleton-based keypoint generation method for grasping objects with roughly uniform height variation,” in 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM). IEEE, 2023, p. 847–853. [23] J. Mahler, F. T. Pokorny, B. Hou, M. Roderick, M. Laskey, M. Aubry, K. Kohlhoff, T. Kröger, J. Kuffner, and K. Goldberg, “Dex-net 1.0: A cloud-based network of 3d objects for robust grasp planning using a multi-armed bandit model with correlated rewards,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016, p. 1957–1964. [24] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, p. 529–533, Feb. 2015.