If you are able to use https instead of ssh, I find "deploy keys" quite handy for this scenario. Gitlab/Github provide these as essentially a temporary https password for selective read-only access to a repo or group of repos with an expiry date. The main downside I find is incompatibility with anything that needs dependencies and expects git over ssh... but then I try to avoid creating the scenario where repos must be built or assembled on each server.

What I would really prefer is to be able to use git over ssh with U2F, i.e a hardware key, in place of a private SSH key this should work the same from the server as the client. U2F is already in openSSH but I am not sure how long it will take before it is commonly available and added to git hosting services... i'm also not sure if the protocol will work through the terminal to an openSSH client on a server.

I already use hardware keys for OTP with SSH (yubico pam) which makes doing ssh between servers secure and easy without private keys, and without client software compatibility issues since it's just a keyboard-interactive mode as far as SSH is concerned... In fact if you are also hosting your own git service you could use this for git cloning over ssh right now and not bother waiting for U2F.