How to Record Robot-Manipulation Data

The thing is, creating robust robot learning models is a real challenge, mainly because we're missing out on large, diverse datasets of real-world manipulation - you know, the kind of data that's hard to come by. Traditional methods, like teleoperating a robot, are not only expensive but also super demanding and tough to scale up. That's where Grabette comes in, a new system that's trying to solve this problem. It's an open, low-cost system that lets users record manipulation data using a handheld gripper, a camera, and a way to recover the 6-DoF trajectory of the hand's movements.

Grabette actually draws inspiration from the Universal Manipulation Interface (UMI) from Stanford, which uses a handheld gripper with a fisheye camera to record demos and recover camera trajectories with SLAM. The Grabette system is made up of a handheld gripper instrumented with two cameras - an RGBD camera for robust 6-DoF tracking, and a wide fisheye camera for context-rich views. What's cool is that the data collected by Grabette can be processed right in a browser and shared on the Hugging Face Hub, making it super easy for anyone to contribute to a shared dataset.

The potential of Grabette is huge - it could make recording manipulation data as simple as shooting a video. This could lead to the creation of a massive, open, collaborative manipulation dataset that no single lab could build on its own. With Grabette, users can record their own manipulation tasks in just minutes, turn them into robot-ready datasets automatically, and help grow the shared dataset. Plus, the system is robot-agnostic, so the same data can be used to drive different robots and learning methods. By making Grabette open source, the developers are hoping to encourage collaboration and progress in the field of robot learning.

Source: Hugging Face