Robots Learning In Realistic Digital Worlds
Robots can now learn to pick, stack, and organize things inside digital rooms without needing real-world training every time.
They may soon learn like people, by practicing in digital worlds rather than physical spaces.
Chatbots like ChatGPT or Claude improve by processing large amounts of text, but robots face a different challenge: they must move through spaces, handle objects, and complete tasks like stacking dishes or setting tables.
Collecting motion data from real robots is slow, costly, and hard to reproduce, while virtual training often looks unrealistic, with objects floating or intersecting.
Researchers at MIT and the Toyota Research Institute have developed steerable scene generation to address this problem.
Without accurate environments, robots cannot learn safely or effectively.
Author's summary: Robots learn in digital worlds.