Robots are good at certain tasks. For example, they are very good at picking up and moving objects and are also getting better at cooking. But while robots can easily complete such tasks in a laboratory, training them in an unfamiliar environment where little data is available is a real challenge.
Announcement
With a new system called OK-Robot, robots could now learn to pick up and move objects on previously unknown terrain. This approach could bridge the gap between the rapid improvement of AI models and the actual capabilities of robots. This means you can avoid additional, expensive and complex training.
To develop the system, researchers from New York University and Meta tested a commercially available robot called Stretch, which comes from the Hello Robot company. It has a simple structure consisting of a wheel, a high pole and a retractable arm. It was used in a total of ten rooms in five apartments, as the researchers describe in their publication, which has not yet been peer-reviewed.
How the robot knows space
To learn about the respective room, a researcher first entered with the robot. The researcher then had to capture the environment using Record3D, an iPhone app that uses the phone’s lidar system, to capture 3D video. He managed to transmit it to the robot.
The OK Robot system then ran an open-source AI object recognition model on the video frames. Combined with other open source models, this helped the robot recognize objects in the room, such as a toy dragon, a tube of toothpaste and a deck of playing cards. But objects in the room that served as a “destination location” could also be recognized in this way: such as a chair, a table and a garbage can.
The task then went like this: the team instructed the robot to pick up a specific object and bring it to a new location. The robot’s pincer arm did this successfully 58.5 percent of the time. The success rate rose to 82% in the least cluttered rooms.
Recommended editorial content
With your consent, an external YouTube video (Google Ireland Limited) will be uploaded here.
Always upload YouTube videos
OK-Robot cleans
The recent boom in artificial intelligence has led to huge leaps in language and image processing capabilities, giving robotics researchers access to open source AI models and tools that didn’t exist three years ago, says Matthias Minderer , senior image processing scientist at Google DeepMind, who was not involved in the project.
“I would say relying entirely on off-the-shelf models is pretty unusual, and making them work is pretty impressive,” he says. “We have experienced a revolution in the field of machine learning that has made it possible to create models that work not only in the laboratory but also in nature,” adds Minderer. “Seeing it actually work in a real environment is very useful information.”
“He only knows these two things”
However, because the researchers used models that were not adapted to this specific project, an impasse was created. If the robot didn’t find the object it was looking for, it simply stopped instead of looking for a solution. This significant limitation is one reason the robot was more likely to succeed in tidier environments: fewer objects meant fewer opportunities for confusion and a clearer space for navigation.
Using ready-made open source templates was both a blessing and a curse, says Lerrel Pinto, an assistant professor of computer science at New York University who co-led the project.
“The good thing is that you don’t need to give the robot additional training data in the environment, it just works,” he says. “The downside is that he can only take one item and put it somewhere else. You can’t ask him to open a drawer. Because he can only do those two things.”
Combining OK-Robot with speech recognition models could allow researchers to give instructions simply by speaking to the robot. This would make it easier for them to experiment with readily available datasets, says Mahi Shafiullah, a graduate student at New York University who co-led the research.
“In the [Robotik-]“There is a feeling in the community that houses are difficult, robots are difficult, and that the combination of houses and robots is completely impossible,” he says. “I think that once people believe that domestic robots are possible, that will happen. “There will be a lot more studies in this area.”
(Jle)
