Allegro Hand Teleop in PyDrake
6.4212 Robotic Manipulation course project
Abstract – This paper presents the design and implementation of a teleoperation scheme for a multi-finger robotic hand in the Drake simulation environment using human demonstrations captured by a single RGB camera. Specifically, teleoperation of the Allegro Hand was implemented using a custom script to obtain joint kinematics of the human hand via a software called MediaPipe. Moreover, through the addition of kinematic hand synergies in this pipeline, we were able to simplify the degrees of freedom used to control the robotic hand. Currently, full teleoperation is incomplete as we were unsuccessful in grasping and manipulating different objects. In the future, we plan to not only address this limitation but to also apply this teleoperation pipeline to conduct behavior cloning of various manipulation tasks based on human demonstration. Such research is important for the design and control of anthropomorphic prosthetic hands.
Below is a video recording of our final presentation.
For further details please see our final report below: