Key Points

• Alibaba's Qwen-Drive 1.0 handles spatial perception, traffic questions, and route planning in one AI model.
• Two modules for 3D mapping and route planning extend the base language model, letting the AI run as both a driving system and a cockpit assistant without losing existing knowledge.
• Retraining cut the rate at which the car veered off the road from 24 percent to 12 percent in simulations. But the model's explanations don't always match the driving decisions it makes.

Qwen-Drive 1.0 handles three tasks in one AI model: spatial perception of the environment, answering questions about traffic, and route planning. The researchers confirm that a text-image model doesn't automatically understand three-dimensional space just because it can describe pictures.

Existing driving models take a general text-image model and fine-tune it on driving data, mostly question-and-answer pairs about traffic situations. According to the paper, this approach has two weaknesses. A model trained mainly on traffic Q&A still can't reliably detect distances, positions, and open spaces. And if it becomes too specialized on driving data, it loses the broad general knowledge from its original training through what the researchers call "catastrophic forgetting," which is exactly the kind of knowledge that matters most in rare, unexpected traffic situations. The model Alibaba's research division built is supposed to address both problems.

A separate module checks whether the model actually understands space

Qwen-Drive-1.0 builds on Qwen3.5-4B, released in February, and adds two extra components. The first generates a bird's-eye-view map of the surroundings by spotting objects in 3D space, figuring out which areas are occupied, and tracing the road layout. The researchers say it doubles as a measuring tool that reveals how much spatial information the model is actually pulling from the images.

The second component, the Planning Expert, uses internal model data to plan the car's future movement. When the researchers trained only the added component and left the vision-language model untouched, spatial accuracy stayed low, confirming that a model capable of describing images in detail doesn't automatically grasp three-dimensional space. Only when the team also trained the vision-language model itself on spatial tasks did performance improve significantly, meaning the ability to spatially understand traffic scenes has to be built in deliberately.

Training starts with the perception module, then combines perception and question answering, followed by route planning. The final step refines the model's behavior through reinforcement learning. For the vision-language component, the team combined 24 publicly available datasets of traffic scenes. These datasets had different structures and sometimes contained errors. An AI model standardized the questions and answers and aligned them with the original data. The team also built its own examples explaining why the car should make a specific driving decision, like which object triggers braking.

One model for both the cockpit and the driving system

In modern vehicles, the infotainment system and the driving system are converging on a single computing unit instead of running on two separate controllers. The authors argue that a model trading general capabilities for pure driving performance doesn't help here, because the cockpit would still need its own model and extra compute for tasks like dialog or open-ended questions.

Qwen-Drive 1.0 scores well above the unmodified base model Qwen3.5-4B on questions about traffic scenes, according to the paper. The biggest gap shows up when the model has to explain cause and effect, like why the car should brake or turn. General knowledge holds up too: The model shows almost no drop on tests outside of driving and even scores slightly higher on some spatial tasks.

From simulation to the road

For driving planning, the team tested the model at several difficulty levels, from simple predictions up to a simulator where errors compound over time. In the simulator, the version retrained with rewards cut the rate at which the car veered off the road from 24 to 12 percent. It also drove more cautiously and covered less distance overall.

The model's explanations don't always pinpoint the actual cause of a situation, though. A red light in the distance and a child stepping into the road call for very different reaction times, and the model can conflate the two. The planned maneuver also doesn't always match the reasoning the model gave beforehand. Some results rest on test procedures the authors designed or rebuilt themselves, so individual metrics say little about how the system would handle messy real-world driving.

The Qwen team measured this gap in spatial understanding itself using its HopChain benchmark. Here, vision-language models misclassified objects and confused spatial relationships even while scoring well on image-text benchmarks. Catastrophic forgetting is a familiar problem, too. When Google Deepmind built PaLM-E in 2023, a model for language, images, and robot control, the smaller variants lost a big chunk of their language ability after robot training. The largest version, at 562 billion parameters, lost almost nothing.

Pairing a language model with a driving function opens up a new attack surface. Researchers at UC Santa Cruz placed a labeled sign in the camera's field of view and tricked the DriveLM driving system into swerving toward crossing pedestrians, even though it had detected them correctly. Research is meanwhile shifting toward World Action Models, which also predict how the environment changes in response to the agent's own actions.

The Qwen team is releasing the model to the research community for free on Hugging Face, ModelScope, and GitHub.