How does AI control robots?

Google DeepMind's latest unveiling - the Gemini Robotics AI model's newest iteration - is a significant leap forward, enabling the control of entire humanoid robots. Previously, the focus was on the upper body, but now Gemini Robotics 2 supports whole-body motions, from toes to fingertips. This means robots can perform a broader range of actions - walking, crouching, stretching, and manipulating objects, for instance. Take Apptronik's Apollo 2 robot, which can now bend down to pick up a watering can or retrieve specific items from a shelf.

The implications are huge for AI practitioners and users, as this update allows robots to tackle more complex, real-world tasks that demand whole-body coordination. Google DeepMind acknowledges there's still room for improvement in movement speed, but this is a major step towards achieving more intricate tasks. And with Gemini Robotics 2, robots also gain better dexterity, courtesy of more complex, five-fingered hands - think sealing a Ziploc or tying a trash bag.

The update also brings improvements to Gemini Robotics ER, a vision-language model that helps robots analyze their surroundings and perform multi-step tasks. Gemini Robotics ER 2 is more adept at completing tasks over an extended period and can now discern when tasks begin and end. It also enables multiple robots of different types to work together seamlessly - like Apollo 2 instructing Google's dual-arm robot to put tools away in a bin while cleaning the garage. Notably, the company says Gemini Robotics ER 2 is its safest model yet, with enhanced detection of nearby humans and automatic safety stops when someone gets too close.

Source: The Verge