A vehicle running UNCAP receives a short text message such as "vehicle merging from the right, about 15 meters away, moving quickly," and treats it as an observation its own sensors could not make. In simulation, this approach to cooperative autonomous vehicles cut communication bandwidth by 63% and raised driving scores by 31% against prior language-based schemes and non-communicating vehicles.
The framework comes from a team led by Neel P. Bhatt, with co-authors including Po-han Li, Sandeep P. Chinchali, David Fridovich-Keil, Zhangyang Wang and Ufuk Topcu. The paper, titled UNCAP: Uncertainty-Guided Neurosymbolic Planning Using Natural Language Communication for Cooperative Autonomous Vehicles, was nominated for the AAMAS 2026 best paper award.
Why language instead of sensor data
The conventional route to vehicle cooperation is to exchange raw camera images, LiDAR point clouds or radar measurements. The UNCAP authors argue this is hard to deploy at scale. Raw streams use large amounts of bandwidth and need heavy computation to process. They also assume every vehicle carries comparable sensing hardware, which will not hold on roads shared by many manufacturers, sensor configurations and software stacks.
Natural language avoids those assumptions. A message like "pedestrian crossing behind bus" is small, does not depend on any particular sensor, and can be read by a vision-language model. The difficulty is that a vehicle at a busy intersection may have ten or twenty neighbors. It has to decide which ones to listen to and which to believe when two descriptions conflict.
The four stages
UNCAP is zero-shot. It trains no new model and instead builds on existing vision-language models, adding a communication layer on top. The process has four stages.
First, each vehicle broadcasts a periodic heartbeat containing only its position and direction of travel. The heartbeat is a few bytes in size and tells nearby vehicles who is around.
Second, a vehicle decides which neighbors matter. It selects those that are close enough and likely to affect its future path. A car moving away from an intersection is filtered out, while one approaching the same intersection is kept. This step reduces unnecessary communication and the computation needed in later stages, which the authors say lets the system handle large teams.
Third, the selected vehicles describe what they see in natural language, along with confidence estimates. Those estimates are calibrated with conformal prediction, a lightweight method intended to make confidence values reflect how reliable the information actually is. The receiving vehicle then measures the mutual information each message contributes. Messages that make the scene clearer receive more weight. Uninformative or potentially misleading ones are discounted.
Fourth, the combined scene description goes to an off-the-shelf vision-language model. The model returns a driving decision such as merge, wait or yield, with a confidence score.
What the tests showed
The team evaluated UNCAP in the CARLA driving simulator using the OPV2V benchmark, which includes highway merging, urban intersections and other scenes where occlusion is common. Compared with non-communicating vehicles and with earlier language-based approaches that broadcast every observation to every car, UNCAP improved both efficiency and safety.
The reported gains were 63% less communication bandwidth, a 31% higher driving score, 61% lower decision uncertainty, and safety margins four times larger during near-miss events.
One result ran against expectation. Sharing raw images performed worse than sharing language. The authors suggest that current vision-language models struggle to reason jointly over several camera feeds taken from different viewpoints. Converting each view into a concise description first appears to act as a form of structured reasoning, distilling the key information before it reaches the decision model.
The improvements held across several underlying models: GPT-4o, GPT-4.1, GPT-4o-mini and GPT-5. The authors read this as evidence that the benefit comes from the communication strategy and not from one model.
Limits and what the result means
The authors state the main limitations. All experiments ran in simulation, so noisy sensing, unreliable wireless links, network delays and packet loss have not been tested. Querying a vision-language model takes roughly a second, so decisions are made at key moments and not continuously.
These limits define how far the result can be read. The 63% bandwidth figure compares UNCAP with schemes that broadcast everything. It shows that selective, calibrated messaging beats indiscriminate messaging. It does not show how the system performs when messages are late, dropped or garbled. Trust weighting based on mutual information is also tested here against simulated neighbors, and the source does not describe how it handles a vehicle that sends false descriptions with high stated confidence.
The raw-image result is the most consequential finding for the field, and also the least settled. If language summaries help because models reason poorly over multiple viewpoints, the advantage could shrink as multi-view models improve. The consistency across four GPT variants suggests the effect is not tied to one model, but all four come from one model family. The source reports no tests on other families.
The design choice that deserves attention is the split between who to talk to and how much to trust them. Earlier cooperative driving work centered on sharing sensor data, which ties vehicles to compatible hardware. A hardware-agnostic text channel with explicit uncertainty fits the mixed-fleet conditions the authors describe. Whether it survives real wireless conditions and latency requirements for continuous control remains unproven. The authors also point to robot fleets, drones and software agents as possible uses, but the paper evaluates only driving. For now, UNCAP is a simulation result for cooperative autonomous vehicles that shows selective language messaging is worth testing on real roads.