> it needs good tool calling, not an encyclopedic knowledge of the world
I wonder if there are any groups/companies out there building something like this
Would love to have models that only know 1 or 2 languages (eg. python + js), but are great at them and at tool calling. Definitely don't need my coding agent to know all of Wikipedia and translating between 10 different languages
1. A special code dataset
2. A bunch of "unrelated" books
My understanding is that the model trained on just the first will never beat the model trained on both. Bloomberg model is my favorite example of this.
If you can squirell away special data then that special data plus everything else will beat the any other models. But that's basically what openai, google, and anthropic are all currently doing.
I wonder if there are any groups/companies out there building something like this
Would love to have models that only know 1 or 2 languages (eg. python + js), but are great at them and at tool calling. Definitely don't need my coding agent to know all of Wikipedia and translating between 10 different languages