I think Apple was actually designing this system internally as an improvement in terms of privacy. Doing Perceptual hashing on the phones is more open and thus auditable than doing the same thing on their servers. They set things up to require multiple different images to match etc.
This kind of technology should not be on phone. Industry standard is doing it on the cloud. It requires only little code changes (let's say in the time frame of 5 years) that law enforcement or whoever says: Please extend that to offline photos and then it's only a few code changes to make that happen. I don't want a ticking time bomb and Apple pinky finger promises that this will not be abused in future.