With large language models, robots can fix their own errors without needing humans

WITH’nin to his new research according to; Large language models (LLM) can help home robots debug without the need for humans. While MIT points out that robots are excellent imitators, unless engineers also program them to adapt to every possible bump or nudge, robots don’t know how to handle these situations other than starting their tasks over again.

The researchers who put forward this new research state that imitation learning, that is, learning to do a task through observation, is popular in the world of home-used robots. However, robots cannot account for countless small environmental variations that can interfere with their smooth operation. Therefore, the system needs to start from the beginning.

The new research considers demonstrations by dividing them into smaller subsets rather than positioning them as part of a continuous action. Large language models also come into play at this point, eliminating the need for the programmer to manually label and assign large numbers of sub-actions.

According to graduate student Tsun-Hsuan Wang, who contributed to the research, by using large language models that explain in natural language how to do each step of a task, a robot will automatically know at what stage it is in a task and can recover by re-planning on its own.

In the demonstration included in the study, a robot is trained to collect marbles and pour them into an empty bowl. This mission actually involves a combination of various smaller tasks for the robots. Large language models stand out with their ability to list and label these subtasks. When the researchers tried to throw the robot off course and knock the marbles out of its spoon to sabotage the mission, the system self-corrected small tasks rather than starting from scratch.

Although this research is only a starting point, it is not difficult to imagine the change that large language models will create in robots in the long term. The research in question is planned to be presented at the International Conference on Learning Representations (ICLR) in May.