I'm going to give a very concrete example of this so people can understand.
I built a fitness product eons ago where there a million rules that determined what should happen for prescribing exercises to athletes (college/pro teams).
If you gave this to an agent today, you will get a tangled mess of if statements that are impossible to debug or extend. This is primarily because LLMs are still bad at picking the right abstraction for a task. The right solution was to build a rules engine, use a constraint solver, and use some combinatorics.
LLMs just don't have the taste to make these decisions without guidance. They also lack the problem solving skills for things they've never seen.*
Was 95% of the app CRUD? Sure. But last I checked, CRUD was never a moat.
*I suspect this part of why senior developers are in extremely high demand despite LLMs.
---
Another example: for many probability problems, Claude loves to code up simulations rather than explore closed form solutions. Asking Claude to make it faster often drives it to make coding optimizations instead of addressing the math. You have to guide Claude to do the right thing, and that means you have to know the right thing to do.
I built a fitness product eons ago where there a million rules that determined what should happen for prescribing exercises to athletes (college/pro teams).
If you gave this to an agent today, you will get a tangled mess of if statements that are impossible to debug or extend. This is primarily because LLMs are still bad at picking the right abstraction for a task. The right solution was to build a rules engine, use a constraint solver, and use some combinatorics.
LLMs just don't have the taste to make these decisions without guidance. They also lack the problem solving skills for things they've never seen.*
Was 95% of the app CRUD? Sure. But last I checked, CRUD was never a moat.
*I suspect this part of why senior developers are in extremely high demand despite LLMs.
---
Another example: for many probability problems, Claude loves to code up simulations rather than explore closed form solutions. Asking Claude to make it faster often drives it to make coding optimizations instead of addressing the math. You have to guide Claude to do the right thing, and that means you have to know the right thing to do.