Instead of storing keys on a pendrive, consider using smart cards or some other crypto devices for that purpose. There are some tradeoffs involved, so research before acting.