Actually driverless cars have the potential to add a vast number of cars to the roads. Today, there's a limit of one car per driver. In the future, one car per same-day package delivery.
True, I'm thinking in a paradigm where humans in cars are replaced with driverless cars. The externalities that you point are valid but I still think that the efficiencies gained by having diverless cars would outpace any externalities (see my lengthy comment about minimum time per car added to a traffic network.)