Right, the thing is, for most tasks, there’s a finish line. Once a model can do a given task right, future improvements may be of little to no benefit beyond cost reduction.
Right now OpenAI and Claude (and maybe Gemini) are over the finish line with reasonable prices for many tasks whereas the rest are still in the race.
But what happens when the open weight models cross the line and it just comes down to price?
At least for now, everything they produce is yours to do with as you please. Just ask them for it. I’ve asked Claude to teach my Hermes agent one skill after another.
Sure you can, but it is really important to set things up so that work is manageable by you. It's not only the skills, it's the capabilities, and secrets, and shared docs, and messaging. Just for fun, these are the agents I have working right now, a mix of Claude Code, Codex and Pi, reported by one of my agents. They coordinate, they do everything that Duet etc can do... but I own the learning:
Juan, across all three machines there are 101 agents. They're in 10 workspaces and 9 aweb teams.
This is the language of someone who is blind to the reality of actual war. I’m sorry, you don’t have PTSD because computers got good at theoretical math problems, and part of the reason half our population looks down on elites is they talk like this to each people who got hit by an IED in Fallujah.
Unless something dramatic changes and it'll become cheap to "constantly re-educate" the models to learn the new fundamental facts about the system (ideally with a small lag of maybe a few days like with humans) I see no reason to think that LLMs can have a profound understanding of how business systems work of a remotely comparable quality to humans. Just adding the context about the system already reduces it's capacity significantly according to what I can tell. Not having large context helps somewhat at a cost of having to explain lots of task-relevant things to LLM over and over again...
> I see no reason to think that LLMs can have a profound understanding of how business systems work of a remotely comparable quality to humans
My manager was dragging me through useless projects that went nowhere despite multiple people telling him "it ain't gonna work" while shooting down proposals that later turned out to be "high priority tasks required directly by upper management". And he's a well-paid, educated employee with years of experience. Don't even get me started on delivery guys who cannot read a simple instruction like "do not ring the bell, leave the package on the doorstep". They just follow a 50-line script and that's all they can do.
Whenever you mention that AI won't reach human intelligence, remember that this is the benchmark. Not Alexander the Great, it's the millions of people throwing their life savings ($1000 each) into Labubus. Long time ago there was a series of articles in my country about the costs of wedding receptions skyrocketing and some people taking out loans to pay that, and someone made a meme "robots will never have human intelligence!" with reply "bro, you took out a loan to pay for a wedding, I don't want your human intelligence".
You are showing disagreements with your priorities (mostly money) as unintelligent. You're showing a lack of care and conscientiousness as unintelligent.
You're reducing intelligence to alignment with money and following instructions. Under that framing the argument is already won in favor of AI amd some vague effective maximizing of a cost function.
Many jobs are like this, so it's fine for employment. Maybe the vast majority of mankind was never suitable for reliable employment of the "cost function" style.
But I know two things:
1. Centralization should be avoided. The reality that AI can do most jobs is not a reason to concentrate power behind those portraying that or building AI. We still want local, ideally individual, empowerment. I don't see enough said about this. Maybe we can even say the system must be one people wish to opt in to, to maximize profit.
2. Intelligence is being very narrowly defined and humans value more than an INT score in the RPG sense. The more I meet people outside my belief circles, the more I realize the intelligence of some actions, in the sense of "logically deduced from a higher principle" not "aligned with my western white collar principles". The ultimate role of culture and humanity IS this principle exploration and to say we've reduced that exploration to "maximize X" is crazy.
At some point deciding fate of the company or system or state flows from principles embedded in cultures and that's in danger of being flattened away by optimizing outcomes with machines.
Vague and wishy washy, but its the clearest I can state it.
> At some point deciding fate of the company or system or state flows from principles embedded in cultures and that's in danger of being flattened away by optimizing outcomes with machines.
If the principle embedded in the culture of my manager is to do dumb shit that goes nowhere, then I'm not sure what goals we're achieving.
I think this could be summarised as alignment being conflated with intelligence - i.e. what seems unintelligent to some is just misalignment with their own values.
Well I'm obviously not comparing to average human intelligence (and negligence), but that's actually a good reminder about one other important aspect of how living organisms are very different from machines -- humans have very high range of intelligence, strength, etc, and the best humans are usually much better than the average in all areas of life. Computers and software are much less diverse.
> The agents only have to be better than we are and we’re not very good. Same argument as self-driving.
Self-driving requires less holistic thinking than imagining, building, and evolving large scale systems. It requires abilities that LLMs do not have. It requires non-symbolic reasoning, intuition, etc. that are beyond LLMs.
I'm surprised this is even a question and the fact that you used human and not even mammal to phrase your question. Do you think intuition is a unique human trait?
"Don't fear change" is a religious statement, no different than "never accept change".
Imagine I am 200IQ, and I can solve things you can't comprehend.
You appear to be saying I should this be able to run any system, including the entire United States fed govt, and anyone who even questions if that may have bad effects should "not fear change".
The industrial revolution was Change, but so is cancer.
Imagine a country where no one can drive, and all transportation relies on a few AI monopolies.
If you don’t pay them, you can walk by feet.
If they ban you, you can walk by feet.
If their infrastructure is damaged, no one and nothing will be able to move.
We have that now in several systems, maybe most, it just isn’t AI. If your water has lead in it, you’ve got one highly incompetent local government to remediate it. If your health care goes out of network with the only hospital system in your small city or town and you have a heart attack you’re fucked. If one security software pushes a bug to production the airlines shut down. Etc.
Single points of failure are a fact of life.
AI offers relative democratization. What’s going on with model competition is giving us more choice, not less.
I'm reminded of the visual novel "Root Double -Before Crime * After Days-" where one guy in town having a manual petrol-powered car instead of using robotaxis is a significant plot point.
You're more than welcome to wake up in a country that is 100% dependent (in all spheres of life and economy!) on a few monopolies. I'm not sure that thing will be a country anymore.
BTW, when (not if, but when) a war happens, they will become very nice targets to render this exact country completely disabled.
I can easily imagine, say, Russia destroying a few datacenters not with nuclear missiles, but with some relatively trivial local ground-based FPV attack or whatever.
And voila. No cargo moved, no code changes or fixes, no medical diagnosis made,etc.
A whole country is disabled.
There are already tons of countries in Europe and Asia that are dependent on a few monopolies. People still live their lives just fine.
Re: war.. that’s why US spends orders of magnitude more on military than anyone else. If any infra is hit on US soil then im heading to the bunker because most of the world will have already been reduced to dust.
There was a book I read maybe 10-15 years ago about a world where governments Essentially dissolved and in place there was nothing but mega corp monopoloys - Jennifer Government was the name ( if you ever played the MUD Nation States yk)
I thought that was an interesting idea, but not something that could ever happen..
Eh, Apple spent years suing Samsung while also buying parts from them for their products. You can be a partner and an enemy at the same time at the high levels of the game.
Animal welfare laws don't give atheist butchers the option to be cruel. They're required to stun the animal first. Why give halal/kosher butchers the option to be cruel? Especially if you claim "most do" choose to stun.
Be kinder to animals, change "most do" to "all do" by removing the religious exception.
...and then come the progressive types to defend 1000+ year old ritual slaughter customs.
In the US at least, freedom of religion does grant some exceptions. The Amish do not participate in Social Security and Medicare, for instance, but I cannot opt out. I’m not sure how it works other places.
reply