Citation:

ZHANG Lingjun, TANG Liang, LIU Lei. Target Position-guided In-hand Re-orientation for Five-fingered Dexterous Hands[J]. ROBOT, 2025, 47(1): 10-21. DOI: 10.13973/j.cnki.robot.240019

Target Position-guided In-hand Re-orientation for Five-fingered Dexterous Hands

ZHANG Lingjun¹,
TANG Liang^{1, 2, ,},
LIU Lei^{1, 2}

1.Beijing Institute of Control Engineering, Beijing 100094, China
2.Science and Technology on Space Intelligent Control Laboratory, Beijing 100094, China

More Information

Abstract

Re-orientation involves rotating an object to a target configuration, with the most challenging case being the rotation from an arbitrary initial configuration to an arbitrary target configuration. To address the challenge of efficiently performing in-hand re-orientation tasks in a more human-like manner by guiding anthropomorphic five-fingered dexterous hands with different degrees of actuation (DoA), a target position-guided in-hand object re-orientation policy generation method is proposed. Firstly, a feasible principle for designing target positions is proposed, inspired by the operation characteristics of human hands during in-hand re-orientation and based on the distribution characteristics of DoA in anthropomorphic five-fingered dexterous hands. The difference between the actual and target positions of the object during re-orientation process is utilized as a component of the immediate reward to guide anthropomorphic five-fingered dexterous hands in maintaining the object near the target. Secondly, a method is developed inspired by the preparatory states of human hands before performing re-orientation tasks, to sample the joint positions of anthropomorphic five-fingered dexterous hands when resetting the state everytime, aiming to enhance manipulation capabilities. Finally, the re-orientation policy is trained using the proximal policy optimization (PPO) algorithm based on the long short-term memory (LSTM) network and asymmetric actor-critic architecture. Simulation results show that the proposed method enables the 9-DoA Schunk SVH dexterous hand, the 13-DoA BICE dexterous hand developed by Beijing Institute of Control Engineering (BICE), and the 18-DoA Shadow dexterous hand to approach the predefined maximum number of consecutive successes when performing re-orientation tasks. Moreover, compared with in-hand object re-orientation policy generation method without target position guidance, the proposed method significantly reduces the average number of steps required to perform re-orientation tasks. The proposed method enables anthropomorphic five-fingered dexterous hands with different DoA to efficiently perform object re-orientation tasks in a human-like manner through coordinated action of the palm and fingers, significantly enhancing operational efficiency.

Keywords:

References

[1]	BICCHI A. Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity[J]. IEEE Transactions on Robotics and Automation, 2000, 16(6): 652-662. doi: 10.1109/70.897777
[2]	KUMAR V, TODOROV E, LEVINE S. Optimal control with learned local models: Application to dexterous manipulation[C]// IEEE International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2016: 378-383. doi: 10.1109/ICRA.2016.7487156
[3]	RAJESWARAN A, KUMAR V, GUPTA A, et al. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations[C]//Robotics: Science and Systems XIV. 2018. doi: 10.15607/RSS.2018.XIV.049
[4]	NAGABANDI A, KONOLIGE K, LEVINE S, et al. Deep dynamics models for learning dexterous manipulation[C]//Proceedings of the Conference on Robot Learning. 2020: 1101-1112. https://proceedings.mlr.press/v100/nagabandi20a.html
[5]	ANDRYCHOWICZ M, BAKER B, CHOCIEJ M, et al. Learning dexterous in-hand manipulation[J]. International Journal of Robotics Research, 2020, 39(1): 3-20. doi: 10.1177/0278364919887447
[6]	CHEN T, XU J, AGRAWAL P. A system for general in-hand object re-orientation[C]//Proceedings of the 5th Conference on Robot Learning. 2022: 297-307. https://proceedings.mlr.press/v164/chen22a.html
[7]	CHEN T, TIPPUR M, WU S, et al. Visual dexterity: In-hand re-orientation of novel and complex object shapes[J]. Science Robotics, 2023, 8(84). doi: 10.1126/scirobotics.adc9244
[8]	HUANG W, MORDATCH I, ABBEEL P, et al. Generalization in dexterous manipulation via geometry-aware multi-task learning[DB/OL]. (2021-11-04) [2024-01-02]. doi: 10.48550/arXiv.2111.03062
[9]	PETRENKO A, ALLSHIRE A, STATE G, et al. DexPBT: Scaling up dexterous manipulation for hand-arm systems with population based training[C]//Robotics: Science and Systems XIX. 2023. doi: 10.15607/RSS.2023.XIX.037
[10]	MA Y J, LIANG W, WANG G, et al. Eureka: Human-level reward design via coding large language models][DB/OL]. (2023-10-06) [2024-03-28]. https://arxiv.org/abs/2310.12931
[11]	KHANDATE G, HAAS-HEGER M, CIOCARLIE M. On the feasibility of learning finger-gaiting in-hand manipulation with intrinsic sensing[C]//International Conference on Robotics and Automation. Piscataway, USA: IEEE, 2022: 2752-2758. doi: 10.1109/ICRA46639.2022.9812212
[12]	KHANDATE G, SHANG S, CHANG E T, et al. Sampling-based exploration for reinforcement learning of dexterous manipulation[DB/OL]. (2023-05-23) [2024-01-20]. https://arxiv.org/abs/2303.03486
[13]	XU Y Z, WAN W K, ZHANG J L, et al. UniDexGrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2023: 4737-4746.
[14]	Shadow Robot Company. Shadow robot dexterous hand] [EB /OL]. (2011-07-25) [2023-12-18]. https://www.shadowrobot.com/dexterous-hand-series/
[15]	Schunk SE & Co. KG. SVH 5-finger hand][EB/OL]. (2022-12-01) [2023-12-19]. http://wiki.ros.org/schunk_svh_driver
[16]	SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[DB/OL]. (2017-08-28) [2023-12-19]. doi: 10.48550/arXiv.1707.06347
[17]	HOCHREITER S, SCHMIDHUBER J. Long short-term memory][J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735
[18]	MAKOVIYCHUK V, WAWRZYNIAK L, GUO Y, et al. Isaac Gym: High performance GPU-based physics simulation for robot learning[DB/OL]. (2021-08-24)[2023-12-19]. https://arxiv.org/abs/2108.10470
[19]	NAIR V, HINTON G E. Rectified linear units improve restricted Boltzmann machines][C]//International Conference on Machine Learning. 2010: 807-814. https://api.semanticscholar.org/CorpusID: 15539264

[1]	FANG Baofu, YU Tingting, WANG Hao, WANG Zaijun. Multi-agent Reinforcement Learning Based on Adaptive State Approximation in Sparse Reward Scenarios[J]. ROBOT, 2024, 46(6): 663-671, 682. DOI: 10.13973/j.cnki.robot.240021
[2]	ZHANG Zhenyi, HUANG Jie. Reinforcement Learning Mission Supervisor Design for Behavior-based Differential Drive Robots[J]. ROBOT, 2024, 46(4): 397-413, 424. DOI: 10.13973/j.cnki.robot.230148
[3]	SHUANG Feng, LU Wanyu, LI Shaodong, YUAN Xiaogang. Robotic Peg-in-hole Assembly Algorithm Based on Reinforcement Learning[J]. ROBOT, 2023, 45(3): 321-332. DOI: 10.13973/j.cnki.robot.220011
[4]	CHEN Jiapan, ZHENG Minhua. A Survey of Robot Manipulation Behavior Research Based on Deep Reinforcement Learning[J]. ROBOT, 2022, 44(2): 236-256. DOI: 10.13973/j.cnki.robot.210008
[5]	DUO Nanxun, LÜ Qiang, LIN Huican, WEI Heng. Step into High-Dimensional and Continuous Action Space: A Survey on Applications of Deep Reinforcement Learning to Robotics[J]. ROBOT, 2019, 41(2): 276-288. DOI: 10.13973/j.cnki.robot.180336
[6]	DU Zhijiang, WANG Wei, YAN Zhiyuan, DONG Wei, WANG Weidong. A Physical Human-Robot Interaction Algorithm Based on Fuzzy Reinforcement Learning for Minimally Invasive Surgery Manipulator[J]. ROBOT, 2017, 39(3): 363-370. DOI: 10.13973/j.cnki.robot.2017.0363
[7]	LI Jun, PAN Qi-shu, HONG Bing-rong. A CBR-Based Multiagent Reinforcement Learning Approach[J]. ROBOT, 2009, 31(4): 320-326.
[8]	DUAN Yong, YANG Huai-qing, CUI Bao-xia, XU Xin-he. Application of Reinforcement Learning to Basic Action Learning of Soccer Robot[J]. ROBOT, 2008, 30(5): 453-459.
[9]	CHEN Wei-dong, XI Yu-geng, GU Dong-lei. A SURVEY OF REINFORCEMENT LEARNING IN AUTONOMOUS MOBILE ROBOTS[J]. ROBOT, 2001, 23(4): 379-384.
[10]	ZHANG Rubo, ZHOU Ning, GU Guochang, ZHANG Guoyin. REINFORCEMENT-LEARNING-BASED OBSTACLE AVOIDANCE LEARNING FOR INTELLIGENT ROBOT[J]. ROBOT, 1999, 21(3): 204-209.

Target Position-guided In-hand Re-orientation for Five-fingered Dexterous Hands

Abstract

References

Related Articles

Login Required