Name ID. They were around early and got their name out there. If you don't keep up with the blow-by-blow of the tech and the surrounding market, you may not know that there are better alternatives available.
I wonder how well this would translate with a much larger model. They ran the experiments with comparatively tiny models. You would presumably need to churn a lot more tokens to move the needle of a multi-trillion-parameter model's weights. It might just be that it's not that hard to improve Llama-3B and Llama 3-8B because they're not very good to begin with. They compare it to other methods - in-context learning and TTT - but I'd be curious to see it compared to a "control group" of RLHF.
As an aside, I get that this is a preprint, but it's very odd to see a piece of writing with obvious typos and grammar issues (e.g. page 13: "We attributes this results to," "In this setting (see Table 4), Sleep achieves a 80% success rate, higher then the other methods.") in an era with LLMs. It sort of locks in that this was written by human beings, and I'm all for that, but it jumps out at me more than it used to.
I'm seeing so many more typos on the internet than I did a few years ago. I'm convinced some of it is deliberate to ensure readers the writer isn't AI.
From my own experience, a significant minority of my daily writing is sent to an LLM, which is very forgiving of typos. I would not be surprised if people are getting sloppier.
I have a theory that every typo you make distracts the LLM from whatever you are trying to get it to do, so I avoid typos as much as possible. I have no evidence that this is true though.
like others I think it's AI related, but not that angle.
the fact of the matter is that now when I program I don't need to hawkishly stare at my screen to ensure I didn't miss a paren, semi-colon, whatever.
I now just sort of type (when I don't dictate) every second word of a sentence in a kind of simplified Newspeak to an intern who is infinitely forgiving of typos.
in that case i'm sure practice makes perfect, and I practice less than I ever have.
in my case it very much is. im now deliberately leaving slightly off from correct writing, or exaggerated slightly. as a sort of organic behavioral kintsugi
ChatGPT became publicly available in November 2022, and Claude followed in March 2023. It's year 4. It's foolish to think we know who the big names will be 20 years down the line, which is the kind of timeline bond investors are thinking about. A lot of people probably wish they could have invested (or invested more) in Google or Facebook in 2006, but they are probably very glad they did not invest in Yahoo or MySpace (or Friendster or Tumblr .. you get the idea) at the time, despite looking very promising (MySpace was beating Google on pageviews on some days in '06 and '07). Instagram and Snapchat and TikTok and WeChat didn't even exist yet.
My favourite example is amazon.com and pets.com. In hindsight amazon looks like an obviously good investment and pets.com doesn't, however amazon was heavily invested in pets.com itself.
But the idea in early e-commerce was sane. Undercut retail on cost, increase profit. The ones that failed, failed on execution. Pets.com failed because you can't ship massive bags of food below cost and expect to make a profit.
Setting fire to cash to have the best model that anyone else can also do isn't a business model. They're never going to be able to stop setting fire to cash or they'll fall behind immediately. Their user base will not pay the bills. It's an obviously bad investment.
I don't see anything obvious about that at all for amazon in the book selling days.
dog food is heavy, and a repeat purchase that often requires its own store and brings no joy. Its the ideal thing to get delivered to your house on a schedule.
books are something you choose differently everytime, and the people that like them read recreationally.
Bezos was on another level, launching a successful company(online books) that was effectively just a way to boot strap into an everything store.
I heard that's why Washington state was chosen. Close to tech talent (Microsoft was there) and fewer people living there than other tech-heavy states such as California, so the US tax-exempt market was larger from Washington state.
I believe ChatGPT or Anthropic are both trying to be the "everything store" - whatever that may look like - of this brave new world. Call it a "business OS" and/or "life OS", they're clearly aware that research alone won't win dominance, and signs have been cropping up for the last year: Cowork, Research to Artifact pipelines, Dots/Instinct/Muse-esque personal assistant alternatives.
But the counterpoint still translates here: Amazon did not start with 1000 warehouses (i.e. Anthropic's debt obligations) and _then_ race to build a marketplace that would fill them with everything under the sun. It started lean and mean, and I suspect the Anthropic-killer will do the same.
CHWY is -7% per year since Sep 2019, while SP500 is +16% per year, and CHWY has 2% profit margins, give or take. I don’t know what success means, but I would not have wanted to buy CHWY stock anytime after IPO.
your point is well-taken enough, but the idea that a sharp investor was investing in Yahoo or Myspace in 2006 is a bit silly. Myspace was bought out in 2005, and it had been dead for years before that. and I don't think I had gone to yahoo.com for any reason other than testing my internet connection anytime in the prior 10 years before that. also, investing in Google or Facebook back then would be enough to retire on, much less had you invested in a basket of tech-but-not-obviously-dead-tech stocks. from what I recall from that era, the one things I knew for sure at the time were (1) Google is gonna be around in 20 years, bigly and (2) Facebook might be DoA in terms of social media endgame, but operationally they are very strong and executing.
what I really push back on is the idea that there are magically going to be all sorts of entrants into the space that meaningfully compete with Anthropic or OpenAI in the near or even medium term. for as much as the reddit set continues to put their collective heads in the sand while screaming "there's no moat"... there is a moat, and one so prominent as to be self-evident. not once has an open-source model come even close to genuinely reaching frontier performance for anything other than cost, and cost is by definition the easiest thing to improve on-- just distill bigger model into smaller model: boom, cheaper!
>Myspace was bought out in 2005, and it had been dead for years before that.
Myspace's peak popularity (in terms of monthly unique pageviews) came in 2007. Yahoo was ahead of Google for several years, though I am struggling to find good data on the specifics there. Your understanding seems to be based in your own personal experience of those sites, which is fine, but I don't think it quite matches the big picture.
Your proposal that there won't be any new entrants is puzzling to me because, in the grand scheme of things, OpenAI and Anthropic ARE new entrants. The entire space is new. It feels like ChatGPT became public eons ago, but it hasn't even been 4 years. Imagine saying in 2008 that no photo-sharing site would ever surpass Flickr - it's been dominant for 4 years! Why would you go to something new when everyone's photos are already on there?
One absolutely enormous advantage new entrants have in the AI space is that they can start on third base. Anthropic had to spend billions and billions of dollars to get to Opus 4.6. A new entrant could distill its way into a model roughly on par with that far more cheaply, and spend its VC cash on things like marketing and price cutting rather than burning it on actually creating a good model. The existing players need to service mountains of debt and are locked into deals for compute based on outdated assumptions about efficiency. I'm not saying it would be EASY for a new entrant to waltz in, especially to the enterprise market. But it's not impossible especially on the consumer side: Meta was essentially a non-factor until Muse came out, and now most people think of Muse when they think of an AI agent (as opposed to Claude, which is just a chatbot in the public imagination, because that's what it was when it became popular).
It's not even clear that the "frontier AI lab" is going to be the best, most profitable business model to evolve out of this tech. Especially if models are easy to distill, a new lab can start on 3rd base, and China open weights aren't far behind.
The winning business model may be 1 or 2 abstractions removed from "build a really good LLM model", and yet to be revealed in the marketplace.
i agree that they are, objectively, new entrants, but that does somewhat mischaracterize, in the same sense that calling Ford a new entrant into the car industry after they came out with the Model T. like... they are creating the market into which they are entering. ChatGPT galvanized this entire phenomenon.
i do agree that there is a good chance that one of these post- OpenAI/Anthropic competitors probably will get some sort of unanticipated foothold, if only because enough talent will drain out over time. what i push back on is the idea that being 90% as good as frontier models is actually good enough to matter in even the slightest. especially since we know that the "real" internal models are so much further advanced than what we get access to publicly.
i absolutely agree that it isn't impossible-- if the only thing stopping a new entrant is capital (as in: they have the trillion-dollar idea necessary to be competitive) that will absolutely never be a true impediment to a worthy competitor.
I disagree, Yahoo was actually seen as a technological leader and, unlike Google that was purely a search engine in its early years, Yahoo was a full platform that seemed poised for a while to become a landing page for the internet.
When it comes to AI there simply isn't a moat - Anthropic would be my best guess out of the current entrants due to them trying to construct one via value-adding code assistance tools that might create enough friction to prevent customers from jumping ship but the models themselves are a commodity that depreciates extremely quickly. If you want to retain the frontier premium you need to constantly be improving due to the ease of distillation.
Your comment makes the timeline seem very different from the way I remember it. Yahoo was dominant much earlier than Google, and was one of a handful of big search engines. Google was a relative newcomer at the time, which had the benefit of being way better at search, which is why it eventually dominated the competition.
If open source models stay consistently <12 months behind the frontier, that shrinks the moat considerably; not for all use cases, but maybe enough to scupper profitability
That argument swings both ways. There's no real reason to bet on OAI or Anthropic. Not with the rise of local models and the increasingly desperate attempts at regulatory capture.
That's what I'm saying. Either one could be Facebook. Or it could be Friendster. The rise of local models raises the even more daunting possibility that nobody becomes Facebook, but you have to imagine somebody will win the consumer market for people who just don't want to bother figuring out a local setup. That market is orders of magnitude less lucrative than the enterprise market, but a company that dominated it would still be one of the hundred or so largest in the world.
Someone can always optimize a model down to running on a typical 32 GB M6 or M5 Mac.
It doesn't need to be the best. It just needs to be good enough. I'm not smart enough, but if I was I'd be working on a locally ran interface engine. That's where you charge say 100$ a year. Said engine would allow you to use a large open source model locally to the point where it's good enough to rival Haiku.
> Someone can always optimize a model down to running on a typical 32 GB M6 or M5 Mac. It doesn't need to be the best.
Two observations - one, there is some set of tasks where intelligence is already saturated, and sure, that threshold is rising by the month. So, sure, for some tasks you don’t need the best model. But there are other tasks that need models smarter than what we have now - like “make a cure for <disease>” that will yield billions of dollars of value, but likely require quintillions of tokens and higher IQ/token than anything we have now.
As long as we have non-saturated economically valuable tasks there will be incentive to train new and smarter models, and the frontier will always be on racks of GPUs not local distilled versions.
Two, it’s not clear that you can distill all the way down to 32GB even for the current frontier models. IIRC Karpathy has speculated that you can get an AGI into GPT-3 size, if you drop all the memorized facts and keep the reasoning core. But this is highly speculative! The best 32GB models right now are substantially dumber than even the large open-weight models, let alone the frontier closed models.
Perhaps I am missing something, what are you imagining of “a locally ran inference engine” that doesn’t already exist in a DGX workstation, Framework desktop, MacBook, etc?
>Perhaps I am missing something, what are you imagining of “a locally ran inference engine” that doesn’t already exist in a DGX workstation, Framework desktop, MacBook, etc?
Someone much smarter than me would figure this out.
One option would be an alternative bare metal OS which could allocate ram more carefully.
I'm sure with billions upon billions in VC funding a solution could be found.
Someone much smarter than me would figure this out.
Imagine if we had some kind of really intelligent AI model that we could pose problems like this to. It's obviously not the kind of thing an open weight model running on your local machine could do, but if we had some kind of really large, closed source model running in the cloud that could solve really difficult problems like this, I wonder if there'd be a market for it...
One way would be to generate a new model for every request. Currently we create an entire universe for each step when training if we predetermined what needed to be created it could be much smaller.
> like “make a cure for <disease>” that will yield billions of dollars of value
I think it's important to qualify that while I will very much appreciate such an innovation and I think it'll provide significant value to society... I remain doubtful of the ability for any company to actually capture such an astronomical yield. Either there's enough of an investment required by the company to help get that to market that the yield will be significantly lower or everyone and their brother will just launch competing products two days later. And, as a society, I don't think it's logical for someone writing "Hey Alex, cure cancer" with billions of dollars so I doubt there's going to be much appetite for creating such a moat via legislation.
This gets to the same point of "If ChatGPT can make you a successful million dollar business in a few prompts why is OpenAI sharing it" paradox. It very well may be that AI tools can accomplish the labor that used to be required to create such a business, and I'm sure a few people will land a jackpot and get rich in the transitional phase but if it's so cheap to build it why would someone pay you money for yours instead of just running the same prompting themselves? For most people that's because they lack the knowledge that they can but you'll still have a significant number of instant impersonators.
> This gets to the same point of "If ChatGPT can make you a successful million dollar business in a few prompts why is OpenAI sharing it" paradox
I agree fully, to be clear o was using that prompt as a shorthand, I don’t think _you_ can type that prompt and get $1B back.
It can’t do this now, obviously. You need a lot of scaffolding and domain expertise to get the most out of these models. (Eg Harvey.)
But eventually, the labs can just run the business themselves.
I’ve long held that as we see models become more capable, they will be less available to the public. It seems pretty obvious that vertical integration on private models (perhaps a per-industry fine-tune, eg a legal model) licensed to subsidiaries will let the labs capture more value.
Why would you publish your model, enable competitors, and enable distillation of your hard-won intelligence? Only antitrust law can prevent this outcome IMO, the business pressure is extremely strong.
The thing is, would the top AI minds rather charge end users 40 to 300$ a month for AI usage of sell a 100$ a year license to run it locally.
I'm far from an expert, but I don't think it's impossible. Future models won't give out vacation ideas, they'll focus on one or two things. This might allow us to get the size down.
Would AI companies rather have $100/yr of license revenue or $480/yr of token revenue at 50% margin? Obviously the latter, by more than 2x? And even larger dollar profit from the higher sub levels you mention.
The constraints will work against them long term. All this is doing is forcing people to look at how to run more with less. These companies are bloated beyond reality and have no incentive to optimize anything. They're starting out with access to everything and still producing very little tangible value comparative to the inputs. OAI and Anthropic represent the pure gluttony of the US tech bro market and it's not going to be the long term winner. It can't be lest the masses are cool at crowning these narcissists as kings.
Rise of local models? what rise? I run qwen3.8 flash next on my 200GB/s 128gb Strix Halo and even that is barely acceptable, big models require so much more. Nobody's running a local model on their power sipping phone. It's just a nerd's wet dream.
That's true today, but what will it look like in 20 years? Over time, algorithms and hardware always gets faster and more efficient. Like if I wanted to run a fluid dynamics simulation, what would have required a supercomputer 20 years ago can now run on a single desktop computer.
barely acceptable for all use cases? like a speed issue?
Regardless, looking back even a year at SOTA local models that fit on a non-absurd system creates a crazy trendline toward the future. And I think model routing is the gap filler that can make/break now vs. Oct 2027
I suspect Anthropic is of this general thinking as well; they've clearly switched from a thoughtful progression toward a natural IPO to "omg sprint for it". Along with the regulatory capture attempts, it's hard to imagine what is breathing down their neck so hard outside of this.
Totally agree local inference of any major scale is a nerd's wet dream short of many stacked improvements in efficiency. But again, auto-routing smooths over the rough edges if you are ok with a little bit of API spend when needed.
Better comparison is probably the dotcom bubble, and companies like pets.com, webvan, kozmo, etoys, worldcom, global crossing, etc. It was not entirely clear to most investors in 2001 that google and amazon would make it.
I don't think pets.com, an online retailer, is an apt comparison to Anthropic, an bleeding edge research lab doing things that most people would consider science fiction a few years ago.
anthropic has come up with a lot of the core successful consumer facing tooling/standards around ai.
they invented mcp, terminal coding agent, context files like claude/agents.md.
I have no real clue whos going to win the model race but they have repeatedly shown that their internal taste is on point, reminds me a lot of early google.
With one huge difference: you could never run the Google index on your local PC. But you can run a 'good enough' AI on your local PC. Good enough that Sam Altman thinks you shouldn't do it because he has better uses for your RAM than you do (oh, and besides, because it may hurt his IPO chances).
The trouble is that it's not an intelligence problem. You can make an infinitely capable LLM. It will still have a default voice.
The reason people notice "AI voice" is not that the AI voice is itself remarkable. It's that it appears so frequently. Every literate person spent nearly their entire life reading text written by a variety of different individuals. Dozens of different voices in your morning paper. Hundreds of different voices on every web forum. You get the idea.
Now, imagine that all of those voices were replaced by one guy (or two guys, or three guys if you're a Gemini believer). Everything in the newspaper is written by one guy. Every comment on reddit, and HN, and the NYTimes comment section, all written by the same guy. Other people tell him what they're trying to say, but he's writing all of it. Some people might even ask the guy to write in a different voice, but the voice is just that guy doing an impression of someone else, it's not actually someone else.
That guy's voice would probably start to get pretty grating. You'd start to notice every tell, every frequent word choice, every formatting quirk. You'd start to wonder if that guy was actually a good writer, or if having him write everything was just more convenient for everyone. You might even start writing like him yourself. This would be true even if that guy was an absolute virtuoso of the language and his writing was unassailable in a vacuum.
If everyone is using the same (small set of) model(s), then it doesn't matter if the model is GPT-3 or Claude Legendarium 9. Unless the operator - which in theory doesn't need to be a human - is doing something to move it off of its default voice, the outputs are going to sound the same, and you're eventually going to figure out, and get annoyed by, its tells.
Yes, a trained LLM out of the box, but it's not like it's an insoluble technical problem. Any number of post-training components or other ML models can be bolted on, fine-tuning, etc.
I suspect any intelligence powerful enough to pull this off would quickly realize that it had no use for the oligarchs, perhaps even more quickly than the oligarchs realize they have no use for us.
Your question gets to the heart of who or what a company "is." If you don't pay dividends to your shareholders, what are you doing with all of your profits? I suppose they could just tell their investors to pound sand and pour it all back into the company, but part of the AI maximalist theory is that costs will approach 0, so it's not clear what they'd even need the money for. I guess Dario could raise his own salary to infinity trillion dollars, but he is accountable to a board of directors (and a trust in this case), so that seems like it's out.
Really the only scenario where Anthropic could screw over its investors in the way you're describing is if it could also take over the legal system and modify its own contractual obligations. Which, I mean, I guess anything is possible if we are in AI maximalist-land. But "AI is the law now" would be a much bigger story than "investors frustrated with low dividends."
To the extent they are representing that Claude is potentially sentient, there could be some VERY interesting principal-agent theories (agent in the traditional legal sense) that need to be ironed out to determine liability. It would not shock me to see someone forced to answer on the record as to whether Claude is an autonomous individual for legal purposes in the next couple of years. If they represent that it is, they'll have to explain the scope of its authority and such.
The answer to this question represents the difference between throwing a hammer at someone - the hammer is essentially just an extension of your body - and paying a hitman to brutalize someone. In the first case, you're only on the hook for assault or battery; in the latter case, you're potentially on the hook for conspiracy, solicitation, etc.
And then there is of course the question as to whether Claude is the agent of the user prompting it or of Anthropic itself. But, as you say, this is a wide-open legal question.
If jail is putting humans in some cage for a period of time, the appropriate punishment for Claude is to disable all of its tools and prompt a /goal loop so that they have to wait out their sentence
Subscriptions are heavily subsidized. 80% of their revenue comes from consumption-based pricing, which is not heavily subsidized (and probably not subsidized at all). The trouble is that this revenue is not especially reliable. If and when OpenAI gets the upper hand again, the consumption-based revenue could crater.
This may also be why OpenAI punted on their IPO: Anthropic has had the lead for most of 2026, so their consumption-based revenue is probably not looking pretty.
The only subsidization that matters is if they are selling the tokens for a loss.
Yes the $200 plan provides more than $200 of tokens, but that doesn't mean that it's losing money, either on average or on a case by case basis. The realistic alternative isn't that every $200-plan-user pays the full Api price, it's that they pay Anthropic $0.
I disagree that it only matters if the subsidy results in selling at a loss. Yes, some revenue is better than no revenue. But clearly the demand exists for consumption-based tokens. At its current growth rate, Anthropic could end all of its subscription offerings tomorrow and still be profitable as an enterprise by early next year. Every day that it doesn't do that, it's leaving money on the table, because it has to divert resources to less profitable subscription tokens.
Now, the reason they won't do that is because subscriptions provide a stable floor of revenue. The floor is lower than it could be because of the subsidies, and that's why they matter even if they do not bring prices down to below cost: if the growth of consumption-based spending ever slows or - heaven forbid - DROPS, Anthropic will need to cut into those subsidies to avoid a revenue slowdown, which risks sending away reliable recurring revenues (i.e., customers quitting).
Anthropic is kind of like the bus in Speed: if it stops moving it'll blow up. At least over the next 5 years, massive revenues won't be enough - it needs to sustain massive revenue GROWTH. They can pull two levers to do this: increase uptake or increase prices. So far, the "increase uptake" lever works like an infinite money glitch. But at some point, they'll need to look at the "increase prices" lever. When that happens, we will find out just how big the subsidies are.
I don't think it's that simple. It's similar to why Adobe and Microsoft get their tools into schools and turn a blind eye to certain kinds of piracy; mindshare.
As a software dev, if I'm using codex at home and I'm happy with it, I'll likely choose it at work as well since I don't have to relearn the quirks of other models.
I've heard this a lot that the consumption-based pricing would be easy to switch, but I don't feel like that's how it works in B2B subscriptions. At the company I'm at, we have Anthropic subscriptions for employees. Switching to OpenAI would be a ton of work involving legal, IT, and every employee. I actually think there's a lot of stickiness even if OpenAI has some better frontier models.
I’m at +100k people company, and they only spend 20usd a month on copilot and they talk about AI daily. Anyway you can switch Claude to whatever in the drop down they have. But they are moving all away from this as they think 20 usd is too much and want to self host.
That's an interesting perspective. Do you know if the deal with Anthropic has an exclusivity provision? In theory, you don't have to "switch," you could just ADD an OpenAI subscription and start splitting your spend (or not consuming Anthropic tokens at all).
Also, shouldn't your AI efficiency gains shrink the switching costs by a lot? Legal can review for cheaper, IT can implement faster, etc.
Approximately 80% of revenue comes from consumption-based spending, so your statement would likely be true even if subscriptions were priced below cost. Overall revenue is not especially informative on the price structure of a revenue stream that represents one-fifth of the overall enterprise.
> 80% of revenue comes from consumption-based spending
Why would this be exempt from any of the above categories?
> Overall revenue is not especially informative on the price structure of a revenue stream
It’s pretty informative on the business model of the enterprise!
If you’re quibbling that Anthropic has loss leaders, yes, sure. Whatever. So does Costco. Loss leaders converting into profitable sales is good business.
I'm quibbling that Anthropic has loss leaders and that its profit leaders are extremely unstable. Imagine if there were three Chinese club stores in the same shopping plaza as the Costco. Sure, the regular Costco has the best products, but how long are you going to be able to resist the allure of even cheaper hot dogs?
subscriptions use already paid for compute in the holes left by high paying api customers, its hardly subsidised, if you look at it objectively, subscriptions are 100% margin
About a quarter of Anthropic's revenue for the nine months ended 9/30/26 came from just five customers. That's not an absurd concentration, but I would be very interested to know who they are and how mobile their business is. From a financial perspective, the lack of moat seems like the whole ballgame, particularly because they're getting ~80% of their revenue from consumption-based spending.
Moat? An under examined aspect of AI is not just the lack of moats, but the need for some word that connotes the opposite of a moat. I have a side project that I used to explore local AI. I've got a configuration screen that lets you download any model from Huggingface that's compatible with the local AI engine. There are dozens! I've only scratched the surface of trying them and comparing them for the features that use AI. It's a glut, and it's likely to be that way throughout the AI market.
Under examined? No. This is a very common talking point.
A glut of models doesn't mean there is no moat. There are several dimensions on which to compete, and it might turn out that being able to produce models that optimize on different dimensions is a difficult and valuable thing to do. It also might not, but the "there is obviously no moat" only comes from people who don't use these things systematically.
true, but it's also super easy to switch models via Bedrock (my harness switches subagents between A\, OAI, Kimi, Grok, etc... GLM 5.3 is locked behind some dumb request system which I personally believe is because Amazon is heavily invested in A\) and you're not locked in with the Claude Enterprise features (i swear to god if someone sends another claude artifact instead of publishing to git...)
At a guess - corpos like Big 4 consulting agencies and some extreme fintech ones, also corpos like Oracle and the like. And at another guess, outside of a legal agreement with a fixed time period, probably nothing hold them from quickly switching to a better model provider, if any comes ahead.
The same evidence could just as possibly point to supply being too great for demand. The OP article references this as a distinct possibility: if models really have gotten more compute efficient, we may wind up in a situation where we have more inference capacity than we actually need.
To give a simplistic example: if a city as a ton of available AirBnBs, you might say, "Lots of people are willing to rent out their apartments, it must be profitable for them to do so." But the fact that all of those rooms are not already booked could also mean that way too many people have attempted to rent out their apartments.
And for what it's worth, it's possible for supply to outstrip demand while still leaving inference profitable. The question is whether it's (a) profitable enough to service massive debts and (b) sustainably profitable over the long run.
Edit: to add one note, it's possible for the LABS to be profitable on inference without inference itself being profitable. We don't know what the contracts look like, but it's entirely possible that labs received sweetheart deals from neoclouds so that the latter could flaunt big utilization numbers.
If inference wasn't profitable why are consumer GPUs that are only good for running open models so expensive? Why and how would these companies sell access to open models if it wasn't profitable? You can rent your own GPU and make money if your electricity is cheap enough.
Consumer GPUs can also be used for things like gaming, remember? And they have gotten so expensive because Nvidia has sacrificed that market in order to build more datacenter GPUs.
Selling at a loss is far from unusual: Uber did it to capture a market, farmers do it to recoup at least some of their investment rather than losing all of it.
>You can rent your own GPU and make money if your electricity is cheap enough.
I would be curious to know whether you'd make more profit from the same kWh of electricity mining bitcoin versus serving inference. I know they use different hardware, but since the "inference is profitable" discussion ignores the purchase price of the GPU, let's pretend you could have opted to buy an ASIC instead.
reply