You're entirely right. But the evidence is not a single drought, it's a staggering combination of many events that show things are changing in a drastic way.
Anyway it's also important to point out that the climate has changed before, but during humanity's existence it never changed this quickly: it looked more like this https://xkcd.com/1732/ which describes changes along thousands of years rather than a century. The derivative of in respect to time is just to steep.
Another thing to consider is that quickly changing environmental conditions is often associated with extinction events. We are in the middle of one such event https://en.wikipedia.org/wiki/Holocene_extinction but maybe one can say it's okay; on a geological scale, such disasters this happens all the time.
The question is, maybe, to ask what will happen to us and the things we care about.
