Magic wormhole and friends are pretty good at doing this securely, especially between servers that can talk to each other without relays.

See also croc