-Policy and Predictability-
The actions of human drivers resulted in over 32,000 traffic fatalities in the United States in 2014. While it’s a staggering number of deaths, automobile safety has been steadily improving with the continual addition of safety features- seatbelts, airbags, reversing cameras and automatic crash avoidance systems.
And now, coming to a highway nearby: cars that drive themselves- or, more specifically, cars that drive us. Where will they be taking us?
Not to the Pentagon…
Automated cars have crashed 14 times since road testing began in live traffic four years ago, and until this spring, the car was not at fault in any of those accidents- a perfect driver.
So why does a perfect driver have an accident? Because it is perfect.
Being perfect, it turns out, is actually something of a problem in the driving world, and I’d daresay the world at large. It seems that obeying the rules 100% of the time makes automated cars discrepant from the rest of traffic in a way that presents unique challenges.
In a demonstration to members of Congress, a self-driving Cadillac SRX SUV was unable to merge onto the I-395 and cross three lanes of traffic to exit for the Pentagon in Washington D.C. Why? It was programmed not to speed, and couldn’t trust that faster drivers would yield. The human driver had to take control in order to make the merge and exit as planned.
Likewise, it has been hard for the Google cars to proceed at four-way stops. The cars were stopping fully and waiting for the other vehicles to do so as well before rightfully proceeding- but the human-driven cars kept inching forward, paralyzing the robot car. Programmers had to adjust the permissions to flex the rules, so that getting through a stop sign wouldn’t require waiting for the street to empty.
Automated cars, programmed for full compliance with the law, struggle to interact with human drivers.
‘A Tricky Set of Circumstances on El Camino Real’
And finally, on February 14 2016, a Google car shared blame in an accident. Having been adjusted to anticipate cooperation from other drivers, it, avoiding sandbags, wrongly assumed that a bus would yield on El Camino Real, and the car’s first potentially at-fault accident took place. Google has referred to the events leading to the accident as “a tricky set of circumstances on El Camino Real”. Indeed.
Total compliance with the law eliminates culpability, but not collisions. Teaching machines the unwritten human agreements expressed with things like eye contact and other subtle cues, is uncharted territory. So too are the moral dilemmas the cars of our future will face: when a collision is unavoidable, whom do you protect? Should the car strike a pedestrian or run off a bridge? What do you do when the rulebook has dissolved, and there are nothing left but values choices? Who will one day make these choices and program these algorithms? Lives, and driving, are replete with many such tricky circumstances.
These concepts are all too familiar to policy writers. Personnel who work among fellow humans and in unpredictable settings will eventually find themselves on the metaphorical I-395, trying to meet an objective. Firm policies tell them to pass the exit- to drive all the way around the world if they have to, and exit at the Pentagon when it’s legal to do so. Experience tells them they can merge across the lanes in safety.
Personnel are the human drivers of our policies. We strive for zero culpability, so we want them to comply with those policies. We also want favourable results, so we ask the human drivers to be ever at the ready for those times when ‘a tricky set of circumstances’ supersedes the rulebook.
We prepare personnel for this huge responsibility by providing instruction in our policies for what to do when exceptions occur, or when circumstances fall outside the provisions of the policy- and by acknowledging that such events will occur. We also do this by clearly communicating and reinforcing our true corporate values, not only in policy but in our everyday operation, such that when, in a split second, decisions are made that were never anticipated in training, our people may grasp the wheel with confidence and steer us to results that we can all live with.
Policy Alchemy 