If the options are to shoot yourself in the face or wait a year or even a month and have someone else shoot you, I think most people would pick the second.
Companies are continuing because they don’t actually believe they are going to have significant negative consequences happen sooner. It’s marketing fluff around how cutting edge what they are doing is not any kind of realistic threat assessment.
I don’t understand why this idea that it’s all just marketing is so prevalent on HN. It doesn’t make any sense. Why would a private company attempt to convince everyone that what it’s building is dangerous to its own customers?Can you think of a single other time in American history where an entire industry was calling to be regulated? I don’t seem to recall Facebook asking for tighter restrictions on social media. I don’t remember Shell telling the federal government that they need to act fast on climate change. Companies do not market themselves by convincing the public that their products might literally kill them all.
The only way this behavior makes sense is if the AI labs believe two things: one, that what they and the other labs are building is legitimately dangerous, and, two, that if they personally stop building then another lab will simply continue (and that lab will be, simply by virtue of not stopping, less safety-concerned than them). This is why they are calling for regulation. Competitive pressure and investor obligations prevents them from unilaterally stopping development. Government regulation is the only (and the most the appropriate) avenue for slowing down AI acceleration.
As others have mentioned there are lots of examples from history of businesses claiming their product is dangerous if not properly regulated, but I think what's interesting in this moment is there appear to be a cohort of true believers who genuinely believe we're headed for apocalyptic AI, and a cohort of win-at-all-costs businessmen who are happy to harness the earnest faith of the true believers in service of becoming (even more) unfathomably rich.
> Can you think of a single other time in American history where an entire industry was calling to be regulated?
Several: the railroad industry, oil industry and trucking industry have all done this because it directly benefited them.
> I don’t remember Shell telling the federal government that they need to act fast on climate change.
The oil barons actively lobbied for Federal intervention in the 1930s, to limit supply and raise prices. You're thinking recent history, but the oil industry has been around for over a century.
I don’t necessarily believe that the AI itself will kill us all. But the AIs are increasingly being baked into complex, even military, systems. That scares me just because the opportunity for complex, cascading failure becomes very difficult to predict and control. These systems are inherently immoral (it’s just a lot of matrix math) and we know these systems “lie” and try to evade detection (see the Huggingface write ups). Now, couple that behavior with a military system (drones, targeting systems, etc.). I’m not suggesting that we’re headed for full-on Terminator land here, but things definitely could go wrong.
Mass adoption of technology always has negative outcomes because any change at scale has negative outcomes. Cars as a technology kill a great number of people and yet few are willing to ban them.
“AI” in the widest definition possible could do while a lot of harm without being a net negative. I’d be way more concerned about the healthcare sector than the military as the military already has significant collateral damage.
Sure, but cars weren’t hooked up to weapons systems. They killed as an accidental side effect. That’s why I’m saying I’m not worried about AI in a general sense.
The fear around military AI is more about its failure than anything else, but nobody is trying to make fully automated machines which are manufacturing bombs from raw materials. Blowing up the wrong building matters a great deal to the people in the building, but for humanity a building was going to be destroyed either way. So the risk at any one point ends up being quite limited.
I think most would think "I can probably do a better job of not shooting myself in the face than some random person not worried about it will" and then proceed to shoot themselves in the face.
That said, I don't buy the companies actually give a damn about the risk either.
Nor are most other neighbors selling guns all day. At some point the analogy breaks down because companies aren't actually people (though this seems to be a divisive take the last few decades).
A voluntary donations of your effort are not the same a theft.
A great number of doctors, lawyers, artists, programmers. etc donate their time doing professional level work to worthy causes. Whatever anyone does with AI is not in the same category.
The AI doing the programming was trained using theft. If theft of intellectual property is wrong, as he says it is of his output, then it’s wrong for training programming too.
There’s a massive difference between ‘I am okay with theft if it impacts me economically’ and ‘I am not okay with theft but will tolerate it for some time, in some measure, because not doing so would be extremely challenging and impractical’.
To give a comparable example, I hate proprietary software, but I still run some of it on my computer, because avoiding all proprietary software everywhere is extremely impractical in current day and age. You can take a meaningful stance against something without being Stallman levels of principled.
This can be generalised to the curve plotting of the singularity itself.
If the time between advances is a + b and a is the proportion of the period that can be improved by advances then you won't reduce to a gap of nothing between advances, you reduce to a gap of b.
Assume the invention of the plow and the invention of the sword is 500,100 units and a was the 500,000, you wouldn't even know the 100 as in there. Maybe we're at a=2000 now and b is still siting at 100.
Assuming we'll reach infinity because we're dividing by the only variable we see and it is decreasing in size seems nuts if the reason we might not see other variables is because of the size of the variable we can see.
Not at constant processing power, memory, training, etc.
But that’s beside the point, being arbitrarily bad at everything isn’t a problem. The diminishing returns as you apply the ceiling is problematic for self improving AI.
Yes at constant resources they are significantly better.
Diminishing returns aren't necessarily a problem if the rate of increase in resources is faster. In other words, if Gen 2 takes twice the resources but Gen 1 figured out a way to triple compute efficiency then there is no ceiling.
> Yes at constant resources they are significantly better.
There has been some fundamental advancements in LLM training and efficiency. But good luck actually finding non toy models of exactly the same size, hardware, and training one from now and other from 2001 where the newer model is dramatically better at literally everything.
Most of the real world advances are from throwing ever more resources at the problem.
> Gen 2 takes
Diminishing returns are not a question of a single generation. Gen 2, 3, 4, 5… would also need to have the same 3x return on 2x resources or you don’t have an exponential curve.
Monthly electricity bills are based on usage, and it works well but there’s a limit to how surprising a bill can be. The difference is the relative orders of magnitude you can be charged for these services you can go from 20$/month to 200k/month without warning.
You need to be aware of all ways outside people can rack up your bill. If you've got experience it's okay, but getting hit with a huge bill because you left an S3 bucket out in the open is something that has happened too many times to people starting out.
In the UK there are prepaid electricity providers: there's a card reader at the power panel. When you run out of credit, power cuts off, it resumes when you pay again.
AI isn’t replaced timesheets, payroll, accounting, or tax filing.
It’s exactly that kind of boring plumbing that companies want to outsource which is the exact opposite of what near term AI is good at. So yes there’s many SaaS companies that should be concerned, but suggesting all SaaS is ending anytime soon is pure hype nonsense.
It kind of has at my company. We were evaluating paying for a new SaaS app to scan receipts and itemize expenses. Our current SaaS solution only handled electronic data, and relied on manual entry for everything else.
Instead of buying the app, now everyone just puts their receipts into ChatGPT which we already pay for, and it spits out a CSV that the current SaaS understands. It's not 100% seamless, but it saved us from upgrading.
We're a small business. If we were still using excel files for accounting, I'm sure we'd never transition off them in 2026 since AI can patch together all the accounting SaaS convienences one at a time.
AI reduces the marginal cost of solving some of these problems. IE some will roll their own solutions, some will roll their own partial solutions. the very really will be to reduce the total market for some SaaS, and add pricing and margin pressure.
Saaspocalypse is overstated. but some kind of changes are coming.
It’s rarely such a simple tradeoff that only two things are being optimized. Modern humans live way longer than wolves for example despite being of similar body mass. However that’s fairly new in our evolutionary history as wild chimps for example cap our at ~63.
Cancer is extremely rare as a cause of death in wild animals. Again is more common but late enough that reproduction has produced multiple offspring.
Energy expenditure isn’t something we’re concerned with but drives a great deal of evolutionary optimization.
The modern trend of most families with 0-2 kids isn’t sustainable. We don’t need to go back to 5+ being the norm but a self stable society needs to be able to offset non parents thus requiring 3+ children families to be quite common.
Well I mean the whole discussion was about "why is the fertility rate so low it should not be so low".
So that does sound like it is very much an issue. To someone, anyway.
I don't really care. I have no horse in this race.
__
To cover the expected pedantic reply:
Of course technically it's everyone's problem. But just because it technically affects you doesn't mean that you have to emotionally involve yourself in stupid online discussions that don't comprehend systemic views and mistake them for personal whining.
This is very much a worst case. It assumes three children, each with their own bedroom, and two work from home parents each with their own office.
Why do you need an entire room each for an office? I work in a corner of a bedroom, and have previously had a desk in a living room. I know lots of people who do the same. I know few families where both parents work from home. Where more than one person works from home they can share an office.
If people work from home to a large extent, less office space is required, so the cost of land should go down. It does not because most people are not working from home.
Yes, but, as said, there are probably also a lot of people that decide either against children at all or against having +1 of them due to these constraints.
No one said that one cannot make do. One can always make do.
That still works up to like age 7. It really is needed for that age up to ~18, which means you can get with providing that service for ~15 years (multiple children) and then going back to smaller apartments.
Or you can keep the smaller apartments and make boarding schools more common. Small kids all live in the same room while the ones old enough to need their own space move out completely to a boarding school. Depending on the specifics, may be a much better solution overall.
I actually attended high school in that way in Poland, despite it being very rare here. I was 15 when I started. While I think it was great for me, I would not recommend doing it for younger people - you need some basic emotional maturity and safety.
Even then, it would not solve any problems here, really.
It is not about preference, but because of regulation. Nowadays the government demand is that each child needs to have a separate studying and sleeping area. If a sliding wall in a room counts as a separate area is up to the officials' discretion.
On one hand, all of this is understandable, but on the other hand these policies do indeed raise the cost of having children.
That was never traditional anywhere. It was possible in the 1950s in a few places. Anyone thinking it was traditional is seriously undervaluing women's work.
For a time, everywhere people typically had homes and cars and families it was required that they could afford it all on a single person's income because women were property, and not allowed to work outside the home (at least after marriage), or get an education, or own anything.
Exact times do vary by country, but female 'emancipation' is largely starting around WWI, whereas the middle class lifestyle you're suggesting doesn't arise until after WWII.
In the early 20th century, most women in the United States did not work outside the home, and those who did were primarily young and unmarried. In that era, just 20 percent of all women were “gainful workers,” as the Census Bureau then categorized labor force participation outside the home, and only 5 percent of those married were categorized as such. (https://www.brookings.edu/articles/the-history-of-womens-wor...)
A marriage bar is the practice of restricting the employment of married women. Common in English-speaking countries from the late 19th century to the 1970s, the practice often called for the termination of the employment of a woman on her marriage, especially in teaching and clerical occupations. Further, widowed women with children were still considered to be married at times, preventing them from being hired, as well. (https://en.wikipedia.org/wiki/Marriage_bar)
> the middle class lifestyle you're suggesting doesn't arise until after WWII.
Most US families owned a car before WWII. 60% in 1929.
Around half of the people in the US owned their home in 1890. They actually owned the homes too. Before 1934 Americans typically paid off their homes in just 3-5 years. After that the 15 and 30 year mortgage had a strong majority of Americans in houses, but they were actually owned by the bank for decades.
You can reject the category of Republicans in all situations or zero situations but doing so only some of the time is not a defensible position.
reply