Does using the Yubikey in this manner protect your keys even if your machine has malware on it? I read through this tutorial [1], and they say you need read/write access to the device, so it seems to me like malware could access your keys while the smartcard is plugged in. If this is the case, I'm not sure how this setup is any more secure than simply keeping your keys on a USB flash drive and plugging it in whenever you need to use ssh.
The private SSH key never leaves the smartcard, not even during authentication. It is not exposed to the OS or any process, at all. You can't extract it at all (maybe the NSA can, who knows). The actual authentication takes place on the token, not in a process on your Unix system.
The only thing that the malware can do is issue an authentication request while the token is plugged in. That's all. If the PIN is not cached, you'll be prompted to enter it, and you'll be like "why is it asking me to enter the PIN?"
Maybe they could run a spy debugger on gpg-agent, but again, this would not give them your private key.
[1] https://blog.habets.se/2013/02/GPG-and-SSH-with-Yubikey-NEO