British autonomous vehicle startup Wayve has unveiled its latest AI system, LINGO-2, which combines language and visual data to enhance autonomous driving capabilities. This vision-language-action model (VLAM) aims to improve driving behavior and decision-making while providing real-time explanations for its actions.
LINGO-2 builds upon the previous LINGO-1 model, released in November 2023, by integrating language with vision capabilities. It processes images from vehicle cameras and road speed rules to provide a continuous commentary on its driving decisions. For example, it can explain why it is slowing down for pedestrians or executing an overtaking maneuver.
During a recent test in central London, LINGO-2 successfully followed a route, changed lanes, adjusted speed according to traffic conditions, safely passed a bus, and stopped at red lights. This successful test marks an important milestone for Wayve in developing advanced autonomous driving systems.
The underlying mechanics of LINGO-2 combine Wayve’s vision model with an auto-regressive language model. The visual model processes camera images into tokens, which are combined with variables like current speed and active route, then fed into the language model to predict driving trajectory and produce commentary text for the car’s controller to act upon.
According to Wayve, models like LINGO-2, which combine visual and language capabilities, “open up new possibilities for accelerating learning with natural language.” They believe that natural language interfaces could enhance user understanding and trust in autonomous driving systems.
Wayve plans to conduct further research on the safety of controlling a car’s behavior with language. After conducting off-road tests in their virtual simulator platform, Ghost Gym, the company aims to conduct more real-world experiments to ensure the safety and reliability of LINGO-2 on the roads.
Since its founding in 2017, Wayve has garnered support from notable backers such as Microsoft, online supermarket Ocado, Virgin Group founder Sir Richard Branson, and the inventor of the internet, Sir Tim Berners-Lee.
