I would bet significant money that, within two years, it will become Generally Obvious that Apple has the best consumer AI story among any tech company.
I can explain more in-depth reasoning, but the most critical point: Apple builds the only platform where developers can construct a single distributable that works on mobile and desktop with standardized, easy access to a local LLM, and a quarter million people buy into this platform every year. The degree to which no one else on the planet is even close to this cannot be understated.
The thing that people seem to have forgotten is that the companies that previously attempted to monetize data center based voice assistants lost massive amounts of money.
> Amazon Alexa is a “colossal failure,” on pace to lose $10 billion this year... “Alexa was getting a billion interactions a week, but most of those conversations were trivial commands to play music or ask about the weather.” Those questions aren’t monetizable.
Google expressed basically identical problems with the Google Assistant business model last month. There’s an inability to monetize the simple voice commands most consumers actually want to make, and all of Google’s attempts to monetize assistants with display ads and company partnerships haven’t worked. With the product sucking up server time and being a big money loser, Google responded just like Amazon by cutting resources to the division.
It doesn't help that Google also keeps breaking everything with the home voice assistants, and this has been true for ages and ages.
I only have a single internet-enabled light in my house (that I got for free), and 90% of the time when I ask the Assistant to turn on the light, it says "Which one?". Then I tell it "the only one that exists in my house", and it says "OK" and turns it on.
Getting it to actually play the right song is on the right set of speakers is also nearly impossible, but I can do it no problem with the UI on my phone.
I don't fear a future where computers can do every task better than us: I fear a future where we have brain-damaged robots annoy the hell out of me because someone was too lazy to do anything besides throw an LLM at things.
> I don't fear a future where computers can do every task better than us: I fear a future where we have brain-damaged robots annoy the hell out of me because someone was too lazy to do anything besides throw an LLM at things.
I had an annoying few weeks where, after years of working properly, Google assistant started misinterpreting "navigate home" as "navigate to the nearest Home Depot™".
QA is the spouses of engineers. Management is a revolving door of the "smartest people" who are thinking about what to eat or their next job. Voices of reason get lost in the noise.
I suspect most engineers at most companies, working behind the scenes to see the sausage being made, grow reservations about recommending it afterwards.
not really limited to their AI products; Android just sometimes randomly decides that pressing play on BT receiver in my car should totally start playing the song directly from my phone instead of the BT it connected to
I'm positive Google's voice commands worked better when Google Home initially released. No idea why it has gotten so bad. Recognition seemed better when it was internal and called "Majel" though on that one I'm sure that's just rose tinted glasses.
It's weird because Gemini is so impressive multimodally but even the Gemini powered assistant can't figure out which lights to turn out, telling the TV to "play" doesn't mean turning it on (it means unpausing it!), just an incredibly frustrating experience.
I feel like you're getting at something different here, but my conclusion is that maybe the problem is the approach of wanting to monetize each interaction.
Almost every company today wants their primary business model to be as a service provider selling you some monthly or yearly subscription when most consumers just want to buy something and have it work. That has always been Apple's model. Sure, they'll sell you services if need be, iCloud, AppleCare, or the various pieces of Apple One, but those all serve as complements to their devices. There's no big push to get Android users to sign up for Apple Music for example.
Apple isn't in the market of collecting your data and selling it. They aren't in the market of pushing you to pick brand X toilet paper over brand Y. They are in the market of selling you devices and so they build AI systems to make the devices they sell more attractive products. It isn't that Apple has some ideologically or technically better approach, they just have a business model that happens to align more with the typical consumers' wants and needs.
> I feel like you're getting at something different here, but my conclusion is that maybe the problem is the approach of wanting to monetize each interaction.
Personally, Google lost me as a search customer (after 25 years) when they opted me into AI search features without my permission.
Not only am I not interested in free tier AI services, but forcing them on me is a good way to lose me as a customer.
The nice thing about Apple Intelligence is that it has an easy to find off switch for customers who don't care for it.
> The nice thing about Apple Intelligence is that it has an easy to find off switch for customers who don't care for it.
Not even only that, but the setup wizard literally asks if you'd like it or not. You don't even have to specifically opt-out of it, because it's opt-in.
Google is currently going full on Windows 10, for 'selected customers', with Gemini in Android. '(full screen popup) Do you want to try out Gemini? [Now] [Later]' 2 hours later... Do you want to...
Yes, there are always ways to deal with companies who make their experience shitty. The point is that you shouldn't have to, and that people will leave for an alternative that doesn't treat them like that.
I feel like this is 5 or so years out of date. The fact that they actually have an Apple Music app for Android is a pretty big push for them. Services is like 25% of their revenue these days, larger than anything except the iPhone.
As I said elsewhere, it really depends on the definition of "service". Subscriptions make up a relatively small minority of that service revenue. For example, 30 seconds of searching suggests that Apple Music's revenue in 2024 was approximately $10b compared to the company as a whole being around $400b. That's not nothing, but it doesn't shape the company in a way that it's competitors are shaped by their service businesses.
The biggest bucket in that "service" category is just Apple's 30% cut of stuff sold on their platform (which it also must be noted, both complements and is reliant on their device sales). That wouldn't really be considered a "service" from either the customer perspective or in the sense of traditional businesses. Operating a storefront digitally isn't a fundamentally different model than operating a brick and mortar store and no one would call Best Buy a "service business".
Call me a naïve fanboy, but I believe that Apple is still one of the very few companies that has an ideologically better approach that results in technically better products.
Where everyone else sells you stuff to make money, they make money to create great stuff.
I know you're saying that Apple's business model is selling devices but it's not like they aren't a services juggernaut.
Where I think you are ultimately correct is that some companies seem to just assume that 100% of interactions can be monetized, and they really can't.
You need to deliver value that matches the money paid or the ad viewed.
I think Apple has generally been decent at recognizing the overall sustainability of certain business models. They've been around long enough to know that most loss-leading businesses never work out. If you can't make a profit from day one what's the point of being in business?
It depends. I guess you can argue this is true purely from scale. However, we should also keep in mind there are a lot of different things that Apple and tech companies in general put under "services". So even when you see a big number under "Service Revenue" on some financial report, we should recognize that most of that was from taking a cut of some other transaction happening on their devices. Relative to the rest of their business, they don't make much from monthly/yearly subscriptions or monetizing their customers' searches/interactions. They instead serve as a middleman on purchase of apps, music, movies, TV, and now even financial transactions made with Apple Card/Pay/Cash. And in that way, they are a service company in the same way that any brick and mortar store is a service company.
I'm confused at what you're trying to say here. Why exactly doesn't the service revenue matter again? For some pedantic reason of Apple being metaphorically similar to a brick and mortar store?
Apple's services revenue is larger than Macs and iPads combined, with a 75% profit margin, compared to under 40% for products (hardware).
Yeah, they serve as a middleman...an incredibly dominant middleman in a duopoly. 80% of teenagers in the US say they have an iPhone. Guess what, all that 15-30% app store revenue is going to Apple. That's pretty much the definition of a service juggernaut.
I also don't agree with you about the lack of selling Apple services to non-Apple users. TV+ is a top-tier streaming service with huge subscriber numbers, and their app is on every crappy off-brand smart TV and streaming stick out there. Yes, there really are Android users who subscribe to Apple Music - 100 million+ downloads on the Google Play store, #4 top grossing app in the music category.
>Why exactly doesn't the service revenue matter again? For some pedantic reason of Apple being metaphorically similar to a brick and mortar store?
You seem to operating under the notion that anything that isn't a device sold is a service. I think that definition is too broad to have any real value and that we should look at the actual business model for a product to determine its categorization. I'm not sure what else to say if you're just going to dismiss that as "pedantic".
But either way, it should be obvious that "services" (however they are defined) are a smaller part of Apple's business than they are for Microsoft, Google, Meta, Twitter, Oracle, Open AI, Anthropic, and most other players in both the general tech and AI spaces.
It's really interesting to consider an area where they are being successful with their AI, the notification summaries work pretty well! It's an easy sell to the consumer bombarded with information/notifications all over the place that on-device processing can filter this and cut out clutter. Basically, don't be annoying. I think a lot of people don't really know how well things like their on-device image search works (it'll OCR an upside-down receipt sitting on a table successfully), I never see them market that strength ever judging by the number of people with iphones that are surprised when I show them this on their own phones.
HOWEVER, you would never know this though given the Apple Store experience! As I was dealing with the board swap in my phone last month, they would have these very loud/annoying 'presentations' every like half hour or so going over all the other apple intelligence features. Nobody watched, nobody in the store wanted to see this. In fact when you consider the history of how the stores have operated for years, the idea was to let customers play around with the device and figure shit out on their own. Store employee asks if they need anything explained but otherwise it's a 'discovery' thing, not this dictated dystopia.
The majority of people I heard around me in the store were bringing existing iphones in to get support with their devices because they either broke them or had issues logging into accounts (lost/compromised passwords or issues with passkeys). They do not want to be told every constantly about the same slop every other company is trying to feed them.
They’ve done loud, in-store presentations for longer than Apple Intelligence has been a thing, but you’re right that it’s a captive audience of mostly disinterested people.
That is exactly why Apple's on-device strategy is the only economically viable one. If every Siri request cost $0.01 for cloud inference, Apple would go bankrupt in a month. But if inference happens on the Neural Engine on the user's phone, the cost to Apple is zero (well, aside from R&D). This solves the problem of unmonetizable requests like "set a timer," which killed Alexa's economics
The greed to lock customers in early on for cheap or free, in hopes to force them on a subscription, absolutely ruined the previous era os assistants. It could have been great with offline inference and foster competition. Instead we got mediocre assistants, thst got worse each year.
The assistant thing really shows the lie behind most of the "big data" economy.
1) They thought an assistant would be able to operate as an "agent" (heh) that would make purchasing decisions to benefit the company. You'd say "Alexa, buy toilet paper" and it would buy it from Amazon. Except it turns out people don't want their computer buying things for them.
2) They thought that an assistant listening to everything would make for better targeted ads. But this doesn't seem to be the case, or the increased targeting doesn't result in enough value to justify the expense. A customer with the agent doesn't seem to be particularly more valuable than one without.
I think that this AI stuff and LLMs in particular is an excuse, to some extent, to justify the massive investment already made in big data architecture. At least they can say we needed all this data to train an LLM! I've noticed a similar pivot towards military/policing: if this data isn't sufficiently valuable for advertising maybe it's valuable to the police state.
> Except it turns out people don't want their computer buying things for them.
I think this also hits an interesting problem with confidence: if you could trust the service to buy what you’d buy and get a good price you’d probably use it more but it only saves a couple of seconds in the easy case (e.g. Amazon reorders are already easy) and for anything less clear cut people rightly worry about getting a mistake or rip-off. That puts the bar really high because a voice interface sucks for more complex product comparisons and they have a very short window to give a high-quality response before most people give up and use their phone/computer instead. That also constrains the most obvious revenue sources because any kind of pay for placement is going to inspire strong negative reactions.
> Those questions aren’t monetizable. ... There’s an inability to monetize the simple voice commands most consumers actually want to make.
There lies the problem. Worse, someone may solve it in the wrong way:
I'll turn on the light in a minute, but first, a word from our sponsor...
Technically, this will eventually be solved by some hierarchical system. The main problem is developing systems with enough "I don't know" capability to decide when to pass a question to a bigger system. LLMs still aren't good at that, and the ones that are require substantial resources.
What the world needs is a good $5 LLM that knows when to ask for help.
This type of response has been given by Alexa from an echo device in my house. I asked, “play x on y”, the response was something like “ok, but first check out this new…”. I immediately unplugged that device and all other Alexa enabled devices in the house. We have not used it since.
This is the monetization wall they have to figure out how to break through. The first inkling of advertising is immediate turn off and destroy, for me.
Even worse than ads, mine keeps trying to jam "News" down my throat. I keep disabling the news feeds on all my devices and they kept re-enabling against my wishes. Every now and then I'll say something to Alexa and she'll just start informing me about how awful everything is, or the echo show in the kitchen will stop displaying the weather in favor of some horrific news story.
Me: "Alexa, is cheese safe for dogs?"
Alexa: "Today, prominent politician Nosferatu was accused by the opposition of baby-cannibal sex trafficking. Nosferatu says that these charges are baseless as global warming will certainly kill everyone in painful ways by next Tuesday at exactly 3pm. In further news, Amazon has added more advertisements to this device for only a small additional charge..."
If I wanted to feel like crap every time I go to the kitchen I'd put a scale in there. /s
I find this a really interesting observation. I feel like 3-4 trivial ways of doing it come to mind, which is sort of my signal that I’m way out of my depth (and that anything I’ve thought of is dumb or wrong for various reasons). Is there anything you’d recommend reading to better understand why this is true?
You are asking why someone don't want to ship a tool that obviously doesn't work? Surely it's always better/more profitable to ship a tool that at least seems to work
GP means they aren't good at knowing when they are wrong and should spend more compute on the problem.
I would say the current generation of LLMs that "think harder" when you tell them their first response is wrong is a training grounds for knowing to think harder without being told, but I don't know the obstacles.
Are you suggesting that when you tell it "think harder" it does something like "pass a question to a bigger system"? I have doubts... It would be gated behind more expensive plan if so
In part because model performance is benchmarked using tests that favor giving partly correct answers as opposed to refusing to answer. If you make a model that doesn't go for part marks, your model will do poorly on all the benchmarks and no one will be interested in it.
Because people make them and people make them for profit. incentives make the product what it is.
an LLM just needs to return something that is good enough for average person confidently to make money. if an LLM said "I don't know" more often it would make less money. because for the user this is means the thing they pay for failed at its job.
> and why that’s a particularly difficult problem to solve
The person I responded to, who seems like someone who definitely knows his stuff, made a comment that implied it was a technically difficult thing to do, not a trivially easy thing that's completely explained by "welp, $$$", which is why I asked. Your comments may point to why ChatGPT doesn't do it, but they're not really answering the actual question, in context.
Especially where the original idea (not mine) was a lightweight LLM that can answer basic things, but knows when it doesn't know the answer and can go ask a heftier model for back-up.
I think that person should think that technically difficult thing that makes more money = gets solved and technically difficult thing that makes less money = doesn't get solved.
By the way model's don't "know". They autocomplete tokens.
> Your comments may point to why ChatGPT doesn't do it
Some features are not meant to be revenue sources. I'd lump assistive technology and AI assistants into the category of things that elevate the usefulness of one's ecosystem, even when not directly monetizable.
Edit: IMO Apple is under-investing in Siri for that role.
Steve Jobs famously said, "If you do the right things on the top line, the bottom line will follow.”
Paraphrased: if you do things with the explicit goal to optimize revenue, it harms your business success. If you do things that optimize user experience and delight customers, it will provide more value long-term.
Voice assistants are in that latter camp, I believe. (And I think of this quote constantly as Tim Cook crams more ads into the ecosystem)
I think of my Alexa often when I think about AI and how Amazon, of all people, couldn't monetize it. What hope do LLM providers have? Alexa is in rooms all around my house and has gotten amazing at answering questions, setting timers, telling me the weather, etc., but would I ever pay a subscription for it? Absolutely not. I wouldn't even have bought the hardware except that it was a loss leader and was like $20. I wouldn't have even paid $100 for it. Our whole economy is mortgaged on this?
This is probably why there’s so much attention on LLM powered coding tools, as it’s one of the few use cases that seem like people would actually pay for it. Ironically mostly developers, who are being marketed as being replaced by AI.
It's also a use case where you already have a user of above-average intelligence who is there correcting hallucinations and mistakes, and is mostly using the technology to speed up boilerplate.
This just doesn't translate to other job types super well, at least, so far.
I'm extremely bearish on AI, but I'm not sure I agree with the framing "not even Amazon could..." All of the advertising around Alexa focused on the simple narrow use cases that people now use it for, and I'm inclined to assume that advertising is part of it. I think another part is probably that voice is really just not that fantastic of an interface for any other kind of interactions. I don't find it surprising that OpenAI's whole framing around ChatGPT, of it being a text-based chat window (as are the other LLMs), is where most of the use seems to happen. I like it best when Alexa acts as a terse butler ("turn on the lights" "done"), not a chatty engaging conversationalist.
Voice assistants that were at the level of a fairly mediocre internet-connected human assistant might be vaguely useful. But they're not. So even if many of us have one or two in our houses or sometimes lean on them for navigation in our cars we mostly don't use them much.
Amazon at one point was going to have a big facility in Boston as I recall focused on Alexa. It's just an uninteresting product that, if it were to go away tomorrow I wouldn't much notice. And I certainly wouldn't pay an incremental subscription for.
This is the part that hasn't made much sense to me. Maybe just.. have a better product?
As you quoted above, "most of those conversations were trivial commands to play music or ask about the weather." Why does any of this need to consume provider resources? Could a weather or music command not just be.. a direct API call from the device to a weather service / Spotify / whatever? Why does everything need to be shipped to Google/Amazon HQ?
From what I can tell, only Apple even wants to try doing any of the processing on-device. Including parsing the speech. (This may be out-of-date at this point, but I haven't heard of Amazon or Google doing on-device processing for Alexa or Assistant.)
So there's no way for them to do anything without sending it off to the datacenter.
> (This may be out-of-date at this point, but I haven't heard of Amazon or Google doing on-device processing for Alexa or Assistant.)
It was out of date 6 years ago.
"This breakthrough enabled us to create a next generation Assistant that processes speech on-device at nearly zero latency, with transcription that happens in real-time, even when you have no network connection." - Google, 2019
Alexa actually had the option to process all requests locally (on at least some hardware) for the first ~10 years, from launch until earlier this year. The stated reason for removing the feature was generative AI.
I had a group of students make a service like this in 2021, completely local, could work offline, did pretty much everything Alexa can do, and they made it connect to their student accounts so they could ask it information about their class schedules. If they can do it, Amazon certainly can. That they don't says they think they can extract more value from monitoring each and every request than they could from selling a better product.
My mother always enjoyed playing Jeopardy! on alexa, it was a novel format and everybody could participate while sitting around and chatting. She happily would have paid for it, even the dreaded monthly subscription, but it was neglected. The service started being buggy (lagging, repeatedly restarting the day's question series) and now they've moved on.
If anyone knows of an open-source alternative I could stitch together, I am all ears!
The difference is previous version of alexa wasn't good enough to pay for it. Now it is good enough that millions of users are paying $10-100 for these services.
Much of the cost of Alexa wasn't the data center costs, as Alexa was not, until recently, an AI. Amazon lost tons of money selling cheap Echo speakers at below cost expecting people would use Alexa on those to buy things. Turns out, people don't like to buy things by yelling at a speaker.
As a sibling poster has said, I don't know how much on-device AI is going to matter.
I have pretty strong views on privacy, and I've generally thrown them all out in light of using AIs, because the value I get out of them is just so huge.
If Apple actually had executed on their strategy (of running models in privacy-friendly sandboxes) I feel they would've hit it out of the park. But as it stands, these are all bleeding edge technologies and you have to have your best and brightest on them. And even with seemingly infinite money, Apple doesn't seem to have delivered yet.
I hope the "yet" is important here. But judging by the various executives leaving (especially rumors of Johnny Srouji leaving), that's a huge red flag that their problem is that they're bleeding talent, and not a lack of money.
I’m much more optimistic on device-side matmul. There’s just so much of it in aggregate and the marginal cost is so low especially since you need to drive fancy graphics to the screen anyway.
Somebody will figure out how to use it—complementing Cloud-side matmul, of course—and Apple will be one of the biggest suppliers.
You don't have to abandon privacy when using an eye - use a service that accesses enterprise APIs, which have good privacy policies. I use the service from the guys who create the This day in AI podcast called smithery.ai -we are access to all of the sota models so we can flip between any model including lots of open source ones within one chat or within multiple chats and compared the same query, using various MCPs and lots of other features. If you're interested have a look at the discord to simtheory.ai (I have no connection to the service or to the creators)
That’s huge. Hope they can continue to keep such people because it isn’t just about one person. It’s all the other smart people that want to work with them.
On-device moves all compute cost (incl. electricity) to the consumer. I.e., as of 2025 that means much less battery life, a much warmer device, and much higher electricity costs. Unless the M-series can do substantially more with less this is a dead end.
That's fair for brute force (running a model on the GPU), but that's exactly where NPUs come in - they are orders of magnitude more energy-efficient for matrix operations than GPUs. Apple has been putting NPUs in every chip for years for a reason. For short, bursty tasks (answer a question, generate an image), the battery impact will be minimal. It's not 24/7 crypto mining, it's impulse load
For the occasional local LLM query, running locally probably won't make much of a dent in the battery life, smaller models like mistral-7b can run at 258 tokens/s on an iPhone 17[0].
The reason why local LLMs are unlikely to displace cloud LLMs is memory footprint, and search.
The most capable models require hundreds of GB of memory, impractical for consumer devices.
I run Qwen 3 2507 locally using llama-cpp, it's not a bad model, but I still use cloud models more, mainly due to them having good search RAG.
There are local tools for this, but they don't work as well, this might continue to improve, but I don't think it's going to get better than the API integrations with google/bing that cloud models use.
What is the possible benefit of on device processing?
I envy your very simple, sedentary life where you are never outside of a high-speed wifi bubble.
Look at almost every Apple ad: It's people climbing rocks, surfing, skiing, enjoying majestic vistas, and all those things that very often come with reduced or zero connectivity.
Battery isn't relevant to plugged-in devices, and in the end, electricity costs roughly the same to generate and deliver to a data center as to a home. The real cost advantage that cloud has is better amortization of hardware since you can run powerful hardware at 100% 24/7 spread across multiple people. I wouldn't bet on that continuing indefinitely, consumer hardware tends to catch up to HPC-exclusive workloads eventually.
You could have an AppleTV with 48 GB VRAM backing the local requests, but... the trend is "real computers" disappearing from homes, replaced by tablets and phones. The advantage the cloud has is Real Compute Power for the few seconds you need to process the interaction. That's not coming home any time soon.
Interestingly, some of Apple’s devices do already serve a special purpose like this in their ecosystem. The HomePod, HomePod Mini, and Apple TV act as Home Hubs for your network, which proxy WAN Apple Home requests to your IoT devices. No other Apple devices can do this.
They also already practice a concept of computational offloading with the Apple Watch and iPhone; more complicated fitness calculations, like VO2Max, rely on watch-collected data, but evidence suggests they’re calculated on the phone (new VO2Max algorithms are implemented when you update iOS, not watchOS)
So yeah; I can imagine a future where Apple devices could offload substantial AI requests to other devices on your Apple account, to optimize for both power consumption (plugged in versus battery) and speed (if you have a more powerful Mac versus your iPhone). There’s good precedent in the Apple ecosystem for this. Then, of course, the highest tier of requests are processed in their private cloud.
If the cloud AI is ad or VC-supported, sure, but that doesn't seem like a sustainable way to provide good user experience.
And don't worry, I'm sure some enterprising electricity company is working out how to give you free electricity in exchange for beaming more ads into your home.
Apple runs all the heavy compute stuff overnight when your device is plugged in. The cost of the electricity is effectively nothing. And there is no impact on your battery life or device performance.
I don't think the throughput of a general purpose device will make a competitive offering; so being local is a joke. All the fun stuff is running on servers at the moment.
From there, AI integration is enough of a different paradigm that the existing apple ecosystem is not a meaningful advantage.
Best case Apple is among the fast copies of whoever is actually innovative, but I don't see anything interesting coming from apple or apple devs anytime soon.
People said the same things about mobile gaming [1] and mainframes. Technology keeps pushing forward. Neural coprocessors will get more efficient. Small LLMs will get smarter. New use-cases will emerge that don't need 160IQ super-intellects (most use-cases even today do not)
The problem for other companies is not necessarily that data center-borne GPUs aren't technically better; its that the financials might never make sense, much like how the financials behind Stadia never did, or at least need Google-levels of scale to bring in advertising and ultra-enterprise revenue.
> All the fun stuff is running on servers at the moment.
With "Apple Intelligence" it looks like Apple is setting themselves up (again) to be the gatekeeper for these kind of services, "allow" their users to participate and earn a revenue share for this, all while collecting data on what types of tasks are actually in high-demand, ready to in-source something whenever it makes economic sense for them...
Outside of fun stuff there is potential to just make chat another UI technology that is coupled with a specific API. Surely smaller models could do that, particularly as improvements happen. If that was good enough what would be the benefit of an app developer using an extra API? Particularly if Apple can offer an experience that can be familiar across apps.
Also why would you want it sucking your battery or heating your room when a data center is only 20 milliseconds away and it's nothing more than a few kilobytes of text. It makes no sense for the large majority of users' preferences which downweight privacy and the ability to tinker.
An LLM on your phone can know everything else that is on your phone. Even Signal chat plaintexts are visible on the phone itself.
People definitely will care that such private data stays safely on the phone. But it’s kind of a moot point since there is no way to share that kind of data with ChatGPT anyway.
I think Apple is not trying to compete with the big central “answer machine” LLMs like Google or ChatGPT. Apple is aiming at something more personal. Their AI goal may not be to know everything, but rather to know you better than any other piece of tech in the world.
And monetization is easy: just keep selling devices that are more capable than the last one.
Gemini can know everything in my Google account, which is basically synonymous with everything that's on my phone, except for text messages. And I use an iPhone. And then Gemini will work just as well on the web when I use my laptop.
So I don't see what unique advantage this gives Apple. These days people's data lives mostly in the cloud. What's on their phone is just a local cache.
I don't know, I feel like Apple shot themselves in the foot selling 8GB consumer laptops up until around 2024 while packing them with advanced AI inference, and usually had lower RAM on their mobile and ipads.
On the other hand all devs having to optimize for lower RAM will help with freeing it up for AI on newer devices with more.
I'd loved to see a strong on-device multi-modal Siri + flexibility with shortcuts.
Besides the "best consumer AI story" they could additionally create a strong offering to SMBs with FileMaker + strong foundation models support baked in. Actually rooting for both!
it will become Generally Obvious that Apple has the best consumer AI story among any tech company.
I love my Macbooks and think they can be great for local LLMs in the future. But the vast majority do not care and they do not want to setup complicated local LLMs. They want something that just works on the computer, tablets, and phones - ideally all synced together.
Local LLMs will never be better than cloud LLMs. They can close the gap if/when cloud LLM progress stalls.
Let's not conflate Apple's failure in cutting edge transformer models with good strategy.
i'd have a lot more respect for apple's "cautious" approach to AI if they didn't keep promising and then failing to deliver siri upgrades (while still calling out to cloud backends, despite all the talk about local LLM), or if they hadn't shipped the absolute trash that is notification summaries.
i think at this point it's pretty clear that their AI products aren't bad because it's some clever strategy, it's bad because they're bad at it. I agree that their platform puts them in a good place to provide a local LLM experience to developers, but i remain skeptical that they will be able to execute on it.
I said "Consumer AI". Even Apple is likely beating Google in consumer AI DAUs, today. Google has the Pixel and gemini.google.com, and that's it; practically zero strategy.
You've never heard of google search (with AI), gmail (with ai), google maps (with ai). Not to mention that most android devices come with google-ified apps that are at worst a click away from Google AI being on the device, in many cases the AI search app is bundled.
No clue how many google smart speakers sold either, but not reason to think it's lower than homepods really, they're 1/3 the price.
Local AI sounds nice but most of Apple’s PCs and other devices don’t come with enough RAM for a decent price needed for good model performance and macOS itself is incredibly bloated.
That's true for current LLMs, but Apple is playing the long game.
First, they are masters of quantization optimization (their 3-4 bit models perform surprisingly well).
Second, Unified Memory is a cheat code. Even 8GB on M1/M2 allows for things impossible on a discrete GPU with 8GB VRAM due to data transfer overhead. And for serious tasks, there's the Mac Studio with 192GB RAM, which is actually the cheapest way to run Llama-400B locally
Depends what you are actually doing. It's not enough to run a chatbot that can answer complex questions. But it's more than enough to index your data for easy searching, to prioritise notifications and hide spam ones, to create home automations from natural language, etc.
Apple has the ability and hardware to deeply integrate this stuff behind the scenes without buying in to the hype of a shiny glowing button that promises to do literally everything.
That might work well for Apple to be the consumer electronic manufacturer that people use to connect to OpenAI/Anthropic/Google for their powerful creative work.
I agree with the assessment that Apple has by far the best platform to ship features.
That being said, if people spend all their time interacting with LLMs for nearly everything, which is the direction we seem to be going in, what locks them in the Apple ecosystem?
Unfortunately, apple will never ditch its luxury brand, so like its memory, even if its good, their business model will never leverage wide spread adoption.
I can explain more in-depth reasoning, but the most critical point: Apple builds the only platform where developers can construct a single distributable that works on mobile and desktop with standardized, easy access to a local LLM, and a quarter million people buy into this platform every year. The degree to which no one else on the planet is even close to this cannot be understated.