A local caching DNS server should offer high-speed response. You could build out a tier if necessary. High-capacity, high-speed DNS is not a greenfield.
The genuine Google requests will typically be cached quickly, and given HTTP codes, you could give a nonpermanent error whilst you're adding new entries, so there's that.
The problem is your misses may come from all over. So they're both expensive and (potentially) large.
Even if Googlebot is only a fraction of requests, it's a large fraction, and I've not investigated how many of same are suspect.
For ASN, either the CIDR Report or Routeviews.org offer information. The latter has downloadable tables, in CIDR format, for IP-based lookups. I've long advocated that this be incorporated into routing hardware with reputation data allowing for linespeed determination of traffic acceptability and rate-limiting. Shifts toward VPN and Tor are making this slightly less useful than previously, but it should still be a generally worthwhile option.
The genuine Google requests will typically be cached quickly, and given HTTP codes, you could give a nonpermanent error whilst you're adding new entries, so there's that.
The problem is your misses may come from all over. So they're both expensive and (potentially) large.
Even if Googlebot is only a fraction of requests, it's a large fraction, and I've not investigated how many of same are suspect.
For ASN, either the CIDR Report or Routeviews.org offer information. The latter has downloadable tables, in CIDR format, for IP-based lookups. I've long advocated that this be incorporated into routing hardware with reputation data allowing for linespeed determination of traffic acceptability and rate-limiting. Shifts toward VPN and Tor are making this slightly less useful than previously, but it should still be a generally worthwhile option.