Elon Musk does a lot of things: Tesla, SpaceX and The Boring Company are three of his day-to-day activities. However, another of his projects is OpenAI, a non-profit research organization looking to advance artificial intelligence. It's sponsored by startup incubator Y Combinator, as well as Microsoft and Amazon Web Services. In a new blog post, the organization has announced it can now train AI to imitate an action after being trained only once in virtual reality.
The AI has been trained to stack blocks on top of each other. The human demonstrator has only demonstrated this to the AI in virtual reality -- the AI never sees it in ever real images before doing it in the physical world for the first time. The way this works is by using two neural networks in tandem to train the AI. First, the "vision network" perceives what it is seeing through its camera, looking at the relation and position of the blocks relative to the robot. Once it knows the positions, the second network, the "imitation network," takes over. It imitates the task by the demonstrator, figures out what the intention of the task at hand is and then stacks the blocks.
The researchers trained the robot on "dozens" of tasks with thousands of demonstrations for each task. These demonstrations are in many different conditions, ranging from color, to lighting to shading. Each demonstration is split into two parts; the first, a complete demonstration by a human, and the second a single observation from the second demonstration. The robot then needs to infer what the demonstrator did in the first demonstration and perform this with no help.
The important thing is that the robot has never done this in reality before; it's only performed the task in VR. It can also infer what it needs to do just from seeing the blocks; it may never have seen the blocks in those positions before, and yet it knows they need to be stacked.
This "imitation learning" is similar to how humans learn, especially as babies and infants. We see another human do something and we do that task too, say drawing, or writing or simply waving or smiling. It's how we learn to walk, talk and do just about everything else in one way or another.
So how could this be useful? For one, it shows that a robot trained in simulation can do a task in real life as well. This means a robot could be trained to do difficult or impossible tasks in simulation and then know how to do the same thing in reality. For researchers, that's an extremely exciting proposition, and one which is sure to fuel continued innovation in this area.