Ah, good point. I hadn't considered that. It looks like you are correct! You might sandwich two Seebeck-Peltier devices and switch between them, though.

https://en.wikipedia.org/wiki/Thermoelectric_effect