You do save some memory and CPU usage, but it appears that the cost is usually higher.

You can disable ABP on particular sites. I was going to suggest doing that for sites you frequently visit that don't have ads... but I just did some quick testing and it doesn't seem to change the memory usage at all.