They can already do this via the cellular phone networks, which need your location to provide the service. Yes, they could discard that data once it was no longer needed, but that is subject to subpoena and preservation orders, so it's not that far to a location history DB anyways.
Mobile network location history is a lot more coarse than what Google can provide. Things like interference and reflections limit the precision most cellular networks can provide even when actively probing for a phone's location. With mmWave things are a lot more precise, but reflection can happen even in mmWave configurations.
