There’s another way besides distillation that’s way cheaper: You can have the big model build prescriptive skills that the small model follows.
Take the “train” portion of tasks on some benchmark, have K3 complete it, and then output detailed descriptions of tools used and why, then run the validation tasks with some small model that has access to the skills.
Take the “train” portion of tasks on some benchmark, have K3 complete it, and then output detailed descriptions of tools used and why, then run the validation tasks with some small model that has access to the skills.