I agree. I am thinking this can only happen if we make training dramatically cheaper and efficient. This is why we are building a new kind of hardware.
Yes. Their architecture recomputes every time so at 150k context every request will have to spend 1.5 min waiting for the model to reread the context.
Say avg model response length is 1024 tok. At 50 tok/s normal providers do your turn will only take 20s (vs Cerebras 101s) and will cost 20x less. That time and cost is per single tool call.
They are increasingly being trained on generated tasks and even (parts) of the pre-training data is 'distilled' (e.g. Clibmix as an open-source example), so there are many ways in which the vocabulary can seep into the model.
reply