I don't understand how this is possible at all at Anthropic. Couldn't they, like, embed an agentic swarm into their backend that prevents any errors from ever making it into production? What am I missing?
I used to talk to strangers a lot when I was younger. But then I started getting older and more scary looking. I developed memories of older men making unwanted advances towards me. I became horribly afraid of making anyone else feel that way, so I stopped.
I know the article's advice is to take a chance, and if I scare someone else so be it. But something about that feels wrong to me.
Reaching middle age, as a guy I thought women were more open to friendliness. I have always assumed it was the shadow of a safe "friendly grandpa" effect. Older men have the opportunity to be seen as less intimidating (assuming you don't emit predator vibes).
Or perhaps alternately I've learnt over the years to be more genuinely friendly.
I've seen men and women attempting to start a friendly conversation and have it backfire - because others can tell if someone is needy. Sometimes people are desperate for a conversation, but they sadly frighten away everyone.
I've also really leant into starting conversations with other guys. The stereotype is a bunch of old men yacking about "boring" stuff, and you can totally just accept that and have fun talking about anything. It's only boring if you lack the wit to discern something interesting within a conversation.
There's also an art to looking approachable, so that others can initiate a conversation with you. I am not skilled at it, but I recognize it. Or alternatively recognizing when someone is open to having a conversation started.
I mean, they definitely expelled Jewish scientists, people like Einstein and Bohr and other prominent physicists, who would have ultimately been very useful to them. Maybe their funding choices would have been different if they hadn't ousted so many researchers.
I read the Oppenheimer biography, so maybe that’s distorting things.
Oppenheimer was Jewish, but very Americanized.
Bohr was working on radar for the Nazis. Einstein had surprisingly little knowledge of nuclear physics, and famously rejected key parts of quantum physics.
The Germans had great nuclear scientists. They simply weren’t willing to spend the money it took to cold the bomb.
I mentioned two as an example, but they expelled significantly more. In 1933 they brought in laws that immediately ousted many Jews from public positions, which caused a mass exodus of intellectuals from the country. James Franck was another prominent physicist who left in protest, and he went on to work directly for the Manhattan Project.
The Nazis drawing a distinction between "Jewish mathematics" and "German mathematics" was also very real.
It's hard to imagine these policies had no effect on their ability to do research, and that it was purely a matter of funding.
"Boris Stoicheff, wrote how the mathematician Edmund Landau was 'physically prevented from entering his classroom by about seventy of his students, some wearing SS uniforms.' They demanded 'German mathematics' instead of 'Jewish mathematics.' One estimate is that the 15% of scientists in Germany who had been fired accounted for about 60% of the country’s physics-based publications."
I really didn't expect OpenAI to do something as immoral as this, despite their history of stealing the world's data to create a public-facing deep-fake generation machine. I am shocked and appalled.
The stories I’ve been reading say that the DoW’s agreement with OpenAI contain the very same limitations as the agreement with Anthropic did. In other words, they pressured Anthropic to eliminate those restrictions, Anthropic declined, then they made a huge fuss calling them “a radical left, woke company,” put them on the supply-chain risk list, then went with OpenAI even though OpenAI isn’t changing anything either.
The whole story makes no sense to me. The DoW didn’t get what they wanted, and now Anthropic is tarred and feathered.
“OpenAI Chief Executive Sam Altman said the company’s deal with the Defense Department includes those same prohibitions on mass surveillance and autonomous weapons, as well as technical safeguards to make sure the models behave as they should.”
The US govt is fighting against true immorality in this very hour, the radical Muslim Iranian government who has been murdering thousands of citizens and holding the population hostage for decades. Ask the Iranian people if they think openai is immoral.
While true, that is not in the top 10 motivations for the US attack. If regime change is successful, the US will be perfectly satisfied with a puppet government that has exactly the same treatment of women, as long as the oil flows.
> The Information had previously reported that $35 billion of Amazon’s investment could be contingent on the company either achieving AGI or making its IPO by the end of the year. OpenAI’s announcement confirms the funding split, but says only that the additional $35 billion will arrive “in the coming months when certain conditions are met.”
So basically, Amazon is buying into the IPO at an early price. Maybe this is the time to divest from MSCI world. I don’t want to be the bag holder in the world’s largest pump and dump.
It can both be true at the same time: That AI is going to disrupt our world and that Open AI does not have a business model that supports its valuation.
yea, proving my point that the index funds are maybe not the safest place if you want to invest into real value. And soon, twitter/Grok/spacex might be doing an IPO
It's this kind of dynamic that makes me pull back on my otherwise pretty AI-forward stance. There's an entire community of people who passionately believe it's obvious and undeniable that Elon Musk has solved problems that he has not solved and his companies deliver things they don't deliver. Tesla is absolutely unambiguous in their marketing material (https://www.tesla.com/fsd) that they do not have autonomous driving, but you're far from the first person I've encountered who's been tricked into believing otherwise.
I don't think that's my relationship with AI, I'm hardly an uncritical booster. But would I know if it was?
You definitely would not know if you were, no one does. I think the healthy position is to assume you are wrong and trying to find evidence of why you are wrong
For instance,I'm very skeptical of AI and, from experience, do not think that the current models are worth the cost, but I'm always in HN trying to find arguments/people that use AI successfully to prove that I'm wrong
I know multiple developers paying thousands of months on AI tooling.
I don't need to convince you it's worth it for you, but it's easy to see that other people have found a way to make it worth it for themselves. I would definitely not spend as much as I personally do if it wasn't worth it to me.
Did it ever occur to you that an entire generation of developers are going to retire in less than 20 years? They are betting that the software industry will be autonomous. Really, think of our industry like AUV phenomena. We’re the drivers that are about to be shown the door, that’s the bet.
World will still need software, lots of it. Their valuation is based on an entire developer-less future world (no labor costs).
Even the rise of high-level languages did not lead to a "developer-less future". What it did was improve productivity and make software cheaper by orders of magnitude; but compiler vendors did not benefit all that much from the shift.
OpenAI has all the name recognition (which is worth a couple billion in itself), but when it comes to actual business use cases in the here and now Anthropic seems ahead. Even more so if we are talking about software dev. But they are valued at less than half of OpenAI's valuation
What is somewhat justifying OpenAI's valuation is that they are still trying for AGI. They are not just working on models that work here and now, they are still approaching "simulating worlds" from all kinds of angles (vision, image generation, video generation, world generation), presumably in hopes that this will at some point coalesce in a model with much better understanding of our world and its agency in it. If this comes to pass OpenAI's value is near unlimited. If it doesn't, its value is at best half what it is today
> What is somewhat justifying OpenAI's valuation is that they are still trying for AGI.
And that's the dealbreaker for me since they've been so adamant on scaling taking them there, while we're all seeing how it's been diminishing returns for a while.
I was worried a few years back with the overwhelming buzz, but my 2017 blogpost is still holding strong. To be fair it did point to ASI where valuation is indeed unlimited, but nowadays the definition of AGI is quite weakened in comparison.. but does that then convey an unlimited valuation?
Obligatory reminder that today's so called "AGI" has trouble figuring out whether I should walk or drive to the car wash in order to get my dirty car washed. It has to think through the scenario step by step, whereas any human can instantly grok the right answer.
The idea/hope is that a video model would answer the car wash problem correctly. There are exactly the kinds of issues you have to solve to avoid teleporting objects around in a video, so whenever we manage more than a couple seconds of coherent video we will have something that understands the real world much better than text-based models. Then we "just" have to somehow make a combined model that has this kind of understanding and can write text and make tool calls
Yes, this is kind of like Tesla promising full self driving in 2016
I just don't know how to engage with these criticisms anymore. Do you not see how increasingly convoluted the "simple question LLMs can't answer" bar has gotten since 2022? Do the human beings you know not have occasional brain farts where they recommend dumb things that don't make much sense?
I should note for epistemic honesty that I expected I would be able to come up with an example of a mistake I made recently that was clearly equally dumb, and now I don't have a response to offer because I can't actually come up with that example.
That problem went viral weeks ago so is no longer a valid test. At the time it was consistently tripping up all the SOTA models at least 50% of the time (you also have to use a sample > 1 given huge variation from even the exact same wording for each attempt).
The large hosted model providers always "fix" these issues as best as they can after they become popular. It's a consistent pattern repeated many times now, benefitting from this exact scenario seemingly "debunking" it well after the fact. Often the original behavior can be replicated after finding sufficient distance of modified wording/numbers/etc from the original prompt.
For example, I just asked ChatGPT "The boat wash is 50 meters down the street. Should I drive, sail, or walk there to get my yacht detailed?" and it recommended walking. I'm sure with a tiny bit more effort, OpenAI could patch it to the point where it's a lot harder to confuse with this specific flavor of problem, but it doesn't alter the overall shape.
This question is obviously ambiguous. The context here on HN includes "questions LLMs are stupid about, I mention boat wash, clearly you should take the boat to the boat wash."
But this question posed to humans is plenty ambiguous because it doesn't specify whether you need to get to the boat or not, and whether or not the boat is at the wash already. ChatGPT Free Tier handles the ambiguity, note the finishing remark:
"If the boat wash is 50 meters down the street…
Drive? By the time you start the engine, you’re already there.
Sail? Unless there’s a canal running down your street, that’s going to be a very short and very awkward voyage.
Walk? You’ll be there in about 40 seconds.
The obvious winner is walk — unless this is a trick question and your yacht is currently parked in your living room.
If your yacht is already in the water and the wash is dock-accessible, then you’d idle it over. But if you’re just going there to arrange detailing, definitely walk."
You can make the argument that the boat variant is ambiguous (but a stretch), it's really not relevant since the point was revealing the underlying failure mode is unchanged, just concealed now.
The original car question is not ambiguous at all. And the specific responses to the car question weren't even concerned with ambiguity at all, the logic was borderline LLM psychosis in some examples like you'd see in GPT 3.5 but papered over by the well-spoken "intelligence" of a modern SOTA model.
I don't understand what occasional hiccups prove. The models can pass college acceptance tests in advanced educational topics better than 99% of the human population, and because they occasionally have a shortcoming, it means they're worse than humans somehow? Those edge cases are quickly going from 1% -> 0.01% too...
"any human can instantly grok the right answer."
When asking a human about general world knowledge, they don't have the generality to give good answers for 90% of it. Even very basic questions humans like this, humans will trip up on many many more than the frontier LLMs.
> If this comes to pass OpenAI's value is near unlimited.
How?
If we have AGI, we have a scenario where human knowledge-based value creation as we know it is suddenly worthless. It's not a stretch to imagine that human labor-based value creation wouldn't be far behind. Altman himself has said that it would break capitalism.
This isn't a value proposition for a business, it's an end of value proposition for society. The only people who find real value in that are people who spend far too much time online doing things like arguing about Roko's Basilisk - which is just Pascal's Wager with GPUs - and people who are so wealthy that they've been disconnected with real-world consequences.
The only reason anyone sees value in this is because the second group of people think it'll serve their self-concept as the best and brightest humanity has ever had to offer. They're confusing ego with ability to create economic value.
"End of human-based value creation" is tantamount to post-scarcity. It "breaks" capitalism because it supposedly obviates the resource allocation problem that the free-market economy is the answer to. It's what Karl Marx actually pointed to as his utopian "fully realized communism". Most people would think of that as a pipe dream, but if you actually think it's viable, why wouldn't you want it?
a) AI is going to replace a Bazillion-Dollar Industry and that
b) being an AI model provider does not allow to capture margins above 5% long-term
I am not saying that this is what will happen, but it's a plausible scenario. Without farmers we would all be dead but that does not mean the they capture monopoly rents on their assets.
There is an interesting AI point here: the US Copyright Office recently tried to argue that images generated by a model could not be copyrighted, no matter how detailed the prompt nor how curated, because the artist did not envision the exact output and thus it is merely the output of an uncreative mechanical process. Clearly OP does not envision the exact way in which glass will shatter or frost or crack, and has to repeatedly update and revise based on what happens; are his glass portraits mere mechanical outputs and uncopyrightable?
I do wonder what the future of using these platforms is in the era of AI. Theoretically, I feel like asking Claude to filter the results to remove "slop" would work quite well. Maybe that could be built into some kind of extension?
I agree. I personally listen to Sam Altman on these types of matters. He's someone with much more extensive qualifications than the pope!
Edit: it looks like I was wrong about this and Sam Altman has no formal qualifications. I still think he has probably picked up a lot of life experience over the years.
reply