Krafcik talks about this very issue in the article on how driver assistance systems give false sense of security to the driver.
> The Waymo team believes its own early experience—when it was the Google self-driving car project—bears that out. In the early 2010s, Google developed a driver-assistance system similar to today's Autopilot and considered selling it to automakers. But when they let Google employees test the software on public roads, they found that drivers came to trust it way too quickly. Drivers who were supposed to be closely monitoring the system instead spent their time looking at their phones, putting on makeup, and other distractions.
> The fundamental challenge here is that the better a driver-assistance system gets, the harder it is to get drivers to pay attention, and the less likely they are to be prepared if the software makes a mistake. The Google team didn't see a good solution to this problem, so they completely changed their strategy. They focused on building a self-driving taxi service that would never have customers in the driver's seat, relying on trained, professional safety drivers to oversee the software during testing.
> The Waymo team believes its own early experience—when it was the Google self-driving car project—bears that out. In the early 2010s, Google developed a driver-assistance system similar to today's Autopilot and considered selling it to automakers. But when they let Google employees test the software on public roads, they found that drivers came to trust it way too quickly. Drivers who were supposed to be closely monitoring the system instead spent their time looking at their phones, putting on makeup, and other distractions.
> The fundamental challenge here is that the better a driver-assistance system gets, the harder it is to get drivers to pay attention, and the less likely they are to be prepared if the software makes a mistake. The Google team didn't see a good solution to this problem, so they completely changed their strategy. They focused on building a self-driving taxi service that would never have customers in the driver's seat, relying on trained, professional safety drivers to oversee the software during testing.