A technique can plan a trajectory for a robot using only language-based inputs. While it can't outperform vision-based approaches, it could be useful in settings that lack visual data to use for training.
A technique can plan a trajectory for a robot using only language-based inputs. While it can't outperform vision-based approaches, it could be useful in settings that lack visual data to use for training.
Rather than encoding visual features from images of a robot's surroundings as visual representations, which is computationally intensive, their method creates text captions that describe the robot's point-of-view. A large language model uses the captions to predict the actions a robot should take to fulfill a user's language-based instructions.
Pan's co-authors include his advisor, Aude Oliva, director of strategic industry engagement at the MIT Schwarzman College of Computing, MIT director of the MIT-IBM Watson AI Lab, and a senior research scientist in the Computer Science and Artificial Intelligence Laboratory ; Philip Isola, an associate professor of EECS and a member of CSAIL; senior author Yoon Kim, an assistant professor of EECS and a member of CSAIL; and others at the MIT-IBM Watson AI Lab and Dartmouth College.
The model repeats these processes to generate a trajectory that guides the robot to its goal, one step at a time. First, because text requires fewer computational resources to synthesize than complex image data, their method can be used to rapidly generate synthetic training data. In one test, they generated 10,000 synthetic trajectories based on 10 real-world, visual trajectories.
In addition, their method could be applied more easily to varied tasks and environments because it uses only one type of input. As long as data can be encoded as language, they can use the same model without making any modifications.
Engineering Optics Spintronics Computer Modeling Artificial Intelligence Robotics Mathematical Modeling
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
SDSU researchers develop AI robot hoping to help those with mental health concernsResearchers at San Diego State University have received millions of dollars in funding that's helped them develop an artificial intelligence robot that they…
Read more »
Researchers invented water-based batteries that store more energy than lithium-ionResearchers have created water-based batteries that can store far more energy than standard lithium-ion batteries.
Read more »
Researchers develop high-performance blue organic LEDs based on thermally activated delayed fluorescence materialOrganic light-emitting diodes (OLEDs) have become a leading display technology. The luminescent material is a core component of OLEDs. Thermally activated delayed fluorescence (TADF) materials have emerged as promising emitters for achieving high-efficiency OLEDs.
Read more »
California-based researchers first to climb world's largest tree in Sequoia National ParkHistory was made as researchers based out of Northern California became the first people to climb one of the world's most famous trees.
Read more »
Researchers develop a roadmap for the development of information technology based on 2D materialsIn a significant stride forward for the semiconductor industry, a new review paper from the 'Shuang-Qing Forum' offers a comprehensive overview of the advancements and strategic roadmap for two-dimensional (2D) materials.
Read more »
Researchers demonstrate the first chip-based 3D printerResearchers have demonstrated the first chip-based 3D printer, a tiny device that emits reconfigurable beams of visible light into a well of resin that rapidly cures into a solid shape. The advance could enable a 3D printer small enough to fit in the palm of a person's hand.
Read more »