Remote inspection and checksum of the device firmware can solve this problem. You should worry about intentional hacking of the firmware, i.e. self-replicating botnet.
This assumes that the firmware contains no bugs. As soon as it does, there is always the possibility of unexpected behaviour and an unintended evolutionary trajectory.
