Once an autonomous vehicle perceives its environment, it must decide what to do. Should it continue straight, change lanes, slow down, or stop? These decisions must be made continuously, quickly, and safely. The planning and decision-making systems that accomplish this are among the most sophisticated components of autonomous vehicles.
The Planning Hierarchy
Autonomous vehicle planning typically operates at multiple levels, from high-level route planning to low-level motion control. Each level addresses different time horizons and decision types.
Route planning determines the path from origin to destination. This is similar to navigation apps—finding the best route considering distance, traffic, and road types. Route planning operates on a time scale of minutes to hours and updates when conditions change significantly.
Behavior planning decides high-level driving actions. Should the vehicle stay in its lane, change lanes, turn at the intersection, or pull over? Behavior planning considers traffic rules, other road users, and the route plan. It operates on a time scale of seconds to minutes.
Motion planning generates the specific trajectory the vehicle will follow. Given a behavior decision like "change lanes," motion planning determines exactly how to execute it—the path, speed profile, and timing. Motion planning operates on a time scale of fractions of a second to seconds.
Control executes the planned trajectory by commanding the vehicle's actuators. Controllers translate the desired motion into steering angles, throttle positions, and brake pressure. Control operates continuously at high frequency.
Predicting Other Road Users
Safe planning requires predicting what other road users will do. A vehicle can't plan a lane change without considering whether the car in the adjacent lane will speed up, slow down, or also change lanes.
Trajectory prediction forecasts where other vehicles and pedestrians will be in the future. Simple predictions might extrapolate current motion—if a car is going straight at 30 mph, it will probably continue doing so. More sophisticated predictions consider context like lane geometry, traffic signals, and typical behavior patterns.
Intent prediction tries to understand what other road users are trying to do. Is that car going to turn left or continue straight? Is that pedestrian going to cross the street? Intent prediction uses cues like turn signals, head orientation, and position relative to crosswalks.
Uncertainty handling acknowledges that predictions are imperfect. Rather than predicting a single future, good prediction systems generate multiple possible futures with associated probabilities. Planning can then consider all likely scenarios rather than betting on a single prediction.
Predicting other road users' behavior is essential for safe planning and decision-making.
Behavior Decision Making
Behavior planning must balance multiple objectives: reaching the destination efficiently, following traffic rules, ensuring safety, and providing comfortable rides. These objectives sometimes conflict, requiring tradeoffs.
Rule-based approaches encode driving knowledge as explicit rules. "If the light is red, stop. If a pedestrian is in the crosswalk, yield. If the lane is ending, merge." Rules are transparent and predictable but may not handle all situations gracefully.
Optimization-based approaches define objectives mathematically and search for actions that optimize them. The system might minimize a cost function that penalizes collision risk, traffic violations, deviation from the route, and passenger discomfort. The action with the lowest cost is selected.
Learning-based approaches train neural networks on examples of good driving behavior. The network learns to map situations to appropriate actions by observing human drivers or through reinforcement learning. These approaches can handle complex situations but are harder to verify and explain.
Most production systems combine these approaches. Rules handle clear-cut situations and enforce safety constraints. Optimization or learning handles more nuanced decisions within the bounds set by rules.
Motion Planning
Once a behavior is decided, motion planning generates the specific trajectory to execute it. This trajectory must be safe, comfortable, and physically achievable by the vehicle.
Path planning determines the geometric path the vehicle will follow. For a lane change, this might be a smooth curve from the current lane to the target lane. The path must avoid obstacles and stay within road boundaries.
Speed planning determines how fast the vehicle moves along the path. The speed profile must respect speed limits, maintain safe following distances, and provide comfortable acceleration and deceleration.
Trajectory optimization combines path and speed planning, optimizing the complete trajectory considering vehicle dynamics, comfort, and safety constraints. The result is a time-parameterized path—the vehicle's position at each moment in time.
Reactive planning handles unexpected situations that require immediate response. If a pedestrian suddenly steps into the road, the system must generate an emergency stop or evasive maneuver quickly, without time for full optimization.
Motion planning generates safe, comfortable trajectories that the vehicle can physically execute.
Handling Uncertainty
Real-world driving involves substantial uncertainty. Perception isn't perfect. Predictions are probabilistic. Other drivers behave unpredictably. Good planning systems account for this uncertainty rather than assuming perfect information.
Robust planning generates plans that work across a range of possible futures. Rather than optimizing for the most likely scenario, robust planning ensures acceptable outcomes even if predictions are wrong.
Contingency planning prepares backup plans for different scenarios. If the car ahead might brake suddenly, the system plans both for continuing normally and for emergency braking. This preparation enables faster response when situations change.
Conservative behavior provides safety margins. Maintaining larger following distances, approaching intersections more slowly, and leaving escape routes all provide buffers against uncertainty. The tradeoff is reduced efficiency.
The Interaction Challenge
Driving involves interaction with other road users who are also making decisions. This creates a complex dynamic where each party's actions depend on what they expect others to do.
Game-theoretic approaches model driving as a game where each participant optimizes their own objectives while considering others' likely responses. These approaches can capture interactive dynamics but are computationally expensive.
Assertive vs. defensive driving represents a key tradeoff. Overly defensive driving—always yielding, never merging into gaps—can be inefficient and even unsafe (blocking traffic, missing opportunities). Overly assertive driving can be aggressive and risky. Finding the right balance is challenging.
Communication helps resolve ambiguity. Human drivers use turn signals, eye contact, and subtle movements to communicate intentions. Autonomous vehicles must both interpret these signals from humans and communicate their own intentions clearly.