How was the latency when you last tried it? Most outboard audio interfaces (even inexpensive ones like the Focusrite Scarletts with ≤ 2 inputs) have a "direct monitor" feature that is as close to zero-latency as you can reasonably get.