Doesn't apply to every scenario where agents are useful. Think about businesses - most business processes are not optimized this way. They are mostly a conglomeration of hierarchy, relationships, customs, IP, etc. Model development will never cross such paths. Thus, harnesses are needed. A foundational system that covers 80% of business processes has value, and I'm targeting that problem --- https://github.com/rush86999/atom
Counter point to “complex businesses” is they’re currently buying Copilot licenses for their employees, with the hopes it can intermingle with Office365. Very ineffective, I think, buy from my exposure the “non-tech” side of things is not on the same timeline as us.
That’s also one of the reasons why I think harnesses might be obsolete in a year, or re-defined completely.
Obviously, once again, I might be completely wrong. But my thesis is investing hard on any model/harness/custom builds might be counter productive given the speed of development. It’s not easy, but having a nimble org where you can swap things around might be a good call.
I think you've really hit the mark on how the harness should be structured:
1. Guardrails - deterministic, social intelligence, team alignment & accountability
2. Learn by doing
3. make it stupid easy for the agent to research and access data
4. DRY
Hi. This is very interesting, could you link to the research? There is a dearth of proper research studies that A/B test what approach is best in terms of harness structure based on repeatable benchmark data with relevant sample uses-cases.
reply