Google DeepMind Puts Gemini AI Into A Robot To Make It A Smarter Navigator Inside The Office

Robots are programmable machines designed to carry out a variety of tasks autonomously or semi-autonomously.

They can range from simple mechanical devices to complex systems with artificial intelligence capabilities.

While they can be equipped with various mechanical complexities to maneuver and such, and various of sensors to understand their surroundings, their intelligence is only limited to the brain inside the robot.

And here, the researchers at Google's DeepMind robotics team said that they're training a robot to navigate its office premises using Gemini 1.5 Pro.

According to the researchers, the AI's "long context window" lets the AI to process and understand vast amounts of information in one go.

Read: Google Gemini 1.5 Pro, With Its 'Long-Context Understanding,' Can Understand Details In Films

Using Large-Language Models like Gemini to robots marks a significant step forward for AI-assisted robots.

With Gemini 1.5 Pro, the researchers at Google DeepMind is able to make the robot process a lot more information than before, in which the robot can "remember" and understand their environment a lot better, making the robot a lot more adaptable and flexible.

"Gemini 1.5 Pro can understand tasks and questions across different modalities because of its long context understanding. When given a 44-minute Buster Keaton film, it's able to find small details in the film and understand plot points," wrote Google in a post on X, during the initial introduction of the AI.

In order to make the robot do what it must do, the team made the it to "watch" video tours of places, just like a person would.

The video in question, includes the layout of the office, showing where thing are, including key features of the space.

Due to Gemini 1.5 Pro having a large memory to begin with, the researchers are able to make the robot tap into this advantage to navigate.

Then, the researchers gave the robot 57 types of tasks to perform in an operating area of around 800 m.

Using Gemini 1.5 Pro, the researchers could simply show the robot a smartphone, and ask, "Where can I charge this?"

With the robot remembering where things are from what it saw in the 'office tour' video, the robot could help lead the user to a power outlet, for example.

Astonishingly, the success rate is a staggering 90%.

This is a big jump in how well robots can get around complex spaces.

The potential applications are endless, from assisting the elderly to enhancing workplace efficiency.

The robots might be able to do even more than just navigate. The DeepMind team has early evidence that these robots can plan out multi-step tasks.

For example, a user with empty soda cans on their desk asks if their favorite drink is in stock.

The robot should then figure out how to navigate the premises to reach the fridge, access the fridge, check for the specific drink it's told to fetch, and return to the user to report what it finds.

This shows a level of understanding and planning that goes beyond simple navigation.

The system shows a lot of promise, but at least at this time, it still requires improvements for real-world application.

For example, it takes the system 10 to 30 seconds to process each instruction. This is extremely slow, and needs speeding up.

Another thing, the researchers only tested the robot in controlled environments, meaning that the robot may not be able to navigate properly in an unpredictable real world, like when things aren't like what it learned.

Like for example, if things are misplaced.