I wonder how long it will take the software industry to re-learn the 2010s lesson, that basing your entire business on (and in this case, firing half of your employees and replacing them with) another company’s API is a bad business decision
Now we just need the RAM market to get back to normal. Or at least fine OpenAI for speculating on raw wafers. There's an article on the front page [0] with this passage that gives me hope that consumer access to VRAM may improve
> On the infrastructure side: OpenAI signed non-binding letters of intent with Samsung and SK Hynix for up to 900,000 DRAM wafers per month, roughly 40% of global output. These were of course non-binding. Micron, reading the demand signal, shut down its 29-year-old Crucial consumer memory brand to redirect all capacity toward AI customers. Then Stargate Texas was cancelled, OpenAI and Oracle couldn’t agree terms, and the demand that had justified Micron’s entire strategic pivot simply vanished. Micron’s stock crashed.
realistically any 'huge' frontier model that takes a rack of H100s to infer against is probably going to have downtime no matter who runs it.
downtime is always going to 'scale' poorly against loads that require a lot of hardware thrown at them, even with lots of good fail-over -- probably worse for the small vendors because they don't have the contracts supplying them with hardware first so availability is already at a premium for them.
so, I guess i'm saying yeah I hope frontier-level-models get out soon in the open arenas, but I suspect the same or similar level of exclusivity will exist as long as they take that much compute to operate.
What makes you say it is a bad business decision? It seems to be a fine decision to make for things like AWS, since when it goes down, a ton of websites go down and no one blames the site.
There is no way to know whether it is a good or bad business decision just because they can go down when a third party goes down. For example, if you save $50 million a year by firing half your employees and replacing them with AI, but you lose $10 million a year because your site goes down when Claude goes down, then you made a great business decision.
Oddly, I do not think you are wrong. In a pure money calculus exercise, this seems like a no brainer. Naturally, the math gets iffy the moment we are trying to capture something less tangible like 'customer may get sufficiently annoyed to drop us altogether' or 'we are no longer a respected company' or what MBAs would call 'unexpected goodwill extraction'.
I honestly don't care nearly as much as I used to, because I used to be more upset over this. Now, I simply wait to see how much is enough to rile up average Joe and Jenna.
On the other hand a competitor site that is up (or bricks and mortar competitors) might get a lot of business when AWS goes down. If you depend on AWS for operations it might be a lot more expensive than that.
Mostly I think its that management does not blame the person who picks AWS. Its another iteration of "no one got fired for buying IBM/Microsoft".
It is also an issue at other levels: if all a county's businesses rely on AWS (let alone its government) then that gives the US huge leverage over you (sanctions would shut down your economy).
This is exactly my point, though. I was simply stating that you can't be sure it is a bad business decision just because it goes down sometimes. It isn't immediately obvious from that single fact whether the business decision is good or bad, it is simply one factor to consider. Occasional downtime isn't an immediate business killer for every business.
That would require AWS to actually be down a lot, and it’s not. Betting your business on AWS being flakier than whatever alternative provider you use is probably not a good idea.
No it would not require that. Suppose you are one of 10 competitors. The others all use AWS.
Your system is down as often as AWS. When you are down your lost sales are shared between 10 of them. When AWS is down you get all their lost sales.
Obviously very simplified, but you get the point. There might be a huge gain in being up when others are down.
> Betting your business on AWS being flakier than whatever alternative provider you use is probably not a good idea.
You are not betting your business on it. You are betting the consequences of downtime only.
AWS does not seem to be all that high reliability out of the box. You can use multiple availability zones etc. but you can do the equivalent elsewhere.
hundreds (thousands?) of companies who based their business on capabilities built around someone else's API. Companies that had important features stop working because a company's API terms and conditions changed. Were you not around for this?
Corporate leaders don't learn lessons. They follow trends, chase growth, reduce the perception of risk, diffuse blame, get their business acquired, and exit with money bags in both hands. No learning from experience necessary.
I am not sure how many people learned it the first time. To be fair, it's really hard to build a business without major dependencies. The key is to assume they will fail and have alternatives available.
Only if you think consolidation of the entire tech industry being funneled into a dozen or so companies as positive sure.
Most of humanity doesn't think this, nor I doubt any devs like the current state of affairs where 4 companies dictate the direction of technology in this country.
Nobody dictates you anything - you’re welcome to setup your own dc if you wish to as some folks do. Not sure about “most of humanity” (lmao) but most of professionals in this line of work clearly don’t think it’s worth it for them
we have a big dependency on AI, both for developers (can survive without it, mostly habits) and internal workflows (very hard to go without it). So we decided to unplug from cloud AI, rent our own GPU and use an open model for both scenarios. We have been very happy with it so far, 60% cheaper and around 50% faster
Faster in what way? All the open models we have access to at work are very noticeably behind the frontier models to the point where it's usually faster to not use them at all.
Is it important to be that self reliant? I wasn't in the workforce then, so I assume if you have an outsourced system with 99% reliability it would be an acceptable risk. Not sure if any AI system will reach that level, but for potential gains it could be worth it.
Former cofounder had a $60 million ARR payment platform on a big company. They saw the success and kicked them off the api and built the same product. Now it’s billions on there public filing income statement.
The rise of the “programmable web,” internet companies offering “free” APIs on which businesses were built, and then destroyed when the company offering the API changed the terms or started charging for it. Twitter was a famous example, Facebook was a major culprit, Google offered a lot of free APIs that wreaked havoc like this, and many, many many smaller forgotten examples.
“Web 2.0” was supposed to be a web of interoperable applications with features exposed by APIs, where you could assemble little pieces of functionality into new and novel configurations. It was cool for a little while, but it was never sustainable. There was an enclosure of the commons (they were never really the commons), and now we’re all digital serfs.
