I imagine the challenge comes from recognizing that your model is trying to call a tool before it actually has and only constraining output then. Running a separate pass for an optionally-empty list of tools afterwards may work, but maybe constraining its output like that causes many spurious tool calls.

Some model providers when using json_schema: true (eg. with_structured_output), it does constrain the output.

constrained decoding tends to make models dumber - this is why it's rarely used