How should driverless cars communicate with pedestrians?
With Waymo's commercial autonomous taxi service kicking off this month, you'd be forgiven for thinking the driverless car conundrum had already been solved; that robo-taxis are here and we'll just have to get used to it.
But getting a car to drive itself — navigating complex roads, following signs, and sharing the lanes safely with other vehicles — is only part of the puzzle. A crucial and arguably more complex, perhaps even more dangerous, aspect is how these vehicles interact with pedestrians.
Read More:
- Waymo gets green light for autonomous cars with no safety driver in California
- Uber robotaxis return to the road after fatality, but with humans in control for now
- How smart voice assistants will help us trust self-driving cars
When a human wishes to cross the road in front of a normal car, they can make eye contact. A wave of the driver's hand quickly and clearly grants permission or indicates caution. These subtle pieces of body language are used every day, by pedestrians, drivers and cyclists alike, as they navigate the urban jungle.
But what happens when the car has no driver? With no human behind the where, there's no face, no hands and no voice. No way of communicating with anyone outside of the vehicle, other than with the turn signals and the actual movement of the car. When an autonomous car stops, is it letting you cross, or is it just working out what to do next?
As Google said in a 2012 patent filing: "Autonomous vehicles lack the capability to directly communicate the vehicle's future behavior." The company also recognizes that by simply stopping, an autonomous car hasn't given a clear enough indication to pedestrians that they are safe to cross the road.
According to patents filed over the last six years, companies developing autonomous cars — including Google (owned by Waymo parent Alphabet), and the ride-sharing firms Lyft and Uber — are all working on vehicle-to-human communication. These documents show how a computer-driven vehicle might tell a pedestrian when it is safe to cross, or to signal that it is the robotic car a customer has hailed from their smartphone, like the taxi driver holding up your name at airport arrivals.
Google filed a patent on driverless car communications back in 2012, before its autonomous vehicle division was spun off into a new company called Waymo. Published in late-2015, the patent is titled Pedestrian Notifications and describes a system for "notifying a pedestrian of the intent of a self-driving vehicle."
The document continues: "For example, the vehicle may include sensors which detect an object such as a pedestrian attempting or about to cross the roadway in front of the vehicle. The vehicle's computer may then determine the correct way to respond to the pedestrian."
After deciding what to do — slow down, stop, or carry on as normal — Google says the vehicle may then "provide a notification to the pedestrian of what the vehicle is going to or is currently doing."
The patent includes drawing which show displays on the front and sides of the car, with messages like "Stop" and "Safe to cross" written on them. The document suggests "a speaker for providing audible notifications" could also be used.
Uber
Ride-hailing company Uber filed a patent for car-to-pedestrian communications in November 2017 and it was published the following March.
Called 'Light Output System for a Self-Driving Vehicle', the patent takes a similar approach to Google but suggests a display with a 'virtual driver', along with written instructions and arrows on a secondary display.
A sketch included with the Uber patent shows an autonomous taxi carrying passengers. The vehicle has an outwards-facing display between the two front seats, showing a computer-generated person pointing in the direction the vehicle wants pedestrians to walk.
There are additional displays on the wing mirrors with arrows pointing in the same direction as the 'virtual driver', plus arrows on the car's front bumper, and a written message which says: "Please process to cross." This way, there should be no confusion among pedestrians over what the car would like them to do. Whether pedestrians will do as they are told, of course, remains to be seen.
Further sketches show how the Uber car could be equipped with a projector at the front, used to beam messages or arrows onto the road ahead; these arrows could indicate the direction the car is about to turn, or be used to tell pedestrians where to walk.
There are yet more displays with arrows fitted ahead of the front wheels and behind the rear wheels, along with displays on all four doors where messages like "Changing lanes" can be shown. These could be seen by pedestrians cyclists and other road users.
Uber's patent explains how an autonomous vehicle like this would need to understand the right-of-way then signal this accordingly to pedestrians. The document states: "If the AV [autonomous vehicle] has right-of-way, the intention signaling system can generate an intention output indicating that the AV will proceed before the humans. This permissive output can also include visual and/or audible feedback (eg flashing red lights or a pedestrian halt symbol on a display)."
Uber even talks about the use of a "mechanical hand" to help show pedestrians where the vehicle plans to go, reminding us of old-fashioned turn signals fitted to vehicles in the early 20th century, before flashing lights became the norm. We somehow doubt this quaint suggestion will become a reality.
Looking at more modern technology, Uber suggests how a vehicle could project a crossing onto the road ahead, guiding a pedestrian's way across the street when it is dark outside.
Lyft
Uber rival Lyft filed a patent for an 'Autonomous Vehicle Notification System' in April 2018, which was published on December 11.
Lyft recognizes that having autonomous cars share the roads with human-driven vehicles "has many challenges," adding: "Drivers and pedestrians are accustomed to interacting in particular ways, removing a driver from some vehicles can lead to uncertainty and miscommunication."
With the patent, Lyft presents a system for helping its customers work out which car is the one they have hailed through the ride-sharing app. Currently, apps like Lyft and Uber give the make, model and license plate of their car in the app, and customers can always check with the driver before getting in.
With autonomous cars, Lyft sees a system where the customer's name is displayed clearly on the car's windshield and side windows, like a taxi driver holding up your name at the airport today.
According to sketches included with the patent, this same notification system can be used to give messages to pedestrians, like "Safe to cross" and arrows directing them where to walk in front of the driverless car.
Other messages like "Yielding" can be shown to let pedestrians and other road users know the car's intention at a junction, while "Safe to pass" can be displayed to cyclists riding alongside a vehicle which plans to slow down or stop.
For now, none of these companies are using the systems outlined in this article, despite all three operating self-driving cars (with human safety drivers) on public roads. Google's Waymo plans to remove its safety drivers soon, but is yet to say if they will be replaced by a system for the car to communicate with other road users.
We suspect vehicles with such a system are likely, if not compulsory, and (once proven to be reliable) will be a key element in convincing the general public that autonomous cars are safe, both for their passengers and others around them.
Trust in such cars is crucial if the industry is to become a success. In May this year data published by AAA claimed 73 percent of American drivers were too afraid to ride in an autonomous vehicle, up from 63 percent the previous December — a drop likely related to the death of a pedestrian hit by an autonomous car being tested by Uber in March.
