To be fair, the whole job market has changed. Layoffs and the death of "a job for life" is not unique to IBM.
I think the pace of progress and innovation has, for better or worse, meant that companies can no longer count on successfully evolving only from the inside through re-training and promotions over the average employee's entire career arc (let's say 30 years).
The reality is that too many people who seek out jobs in huge companies like IBM are not looking to constantly re-invent themselves and learn new things and keep pushing themselves into new areas every 5-10 years (or less), which is table stakes now for tech companies that want to stay relevant.
Honestly, I think that's people reacting to the market more than it's the market reacting to people.
If your average zoomer had the ability to get a job for life that paid comparably well by a company that would look after them, I don't think loyalty would be an issue.
The problem is today, sticking with a company typically means below market reward, which is particularly acute given the ongoing cost of living crises affecting the west.
Well, if you're unable to read, you're not going to figure out what the buttons do by reading the textual labels :p
Further, if you have difficulty reading, it's easier to parse the meaning of an abstract symbol, so you'd use that instead of a textual label when available. (I say this as someone who is a really slow reader. I use icons when I can)
> Screen size makes little difference for an individual they can just sit closer
This is silly. Most people don’t want to sit in a chair 3 feet from their TV to make it fill more of their visual area. A large number of people are also not watching movies individually. I watch TV with my family far more than I watch alone.
Tell that to every streaming on their tablets sitting on their stomachs. People even watch movies on their phones but they aren’t holding them 15’ away.
No one says the experience of watching on their tablet matches the experience of watching a movie in the theater.
But this isn’t the point. TVs are furniture. People generally have a spot where the TV naturally fits in the room regardless of its size. No one buys a TV and then arranges the rest of their furniture to sit close enough to fill their visual space. If the couch is 8 feet from the TV, it’s 8 feet from the TV.
People watching their tablet on a couch in from of a 55+” TV with a surround sound speaker system says on some level it’s a better experience. I’ve seen plenty of people do this to say it’s common behavior.
> No one buys a TV and then arranges the rest of their furniture to sit close enough to fill their visual space. If the couch is 8 feet from the TV, it’s 8 feet from the TV.
It’s common on open floor plans / large rooms for a couch to end up in a completely arbitrary distance from a TV rather than next to a wall. Further setting up the TV on the width vs length vs diagonal of a room commonly provides two or more options for viewing distance.
> People watching their tablet on a couch in from of a 55+” TV with a surround sound speaker system says on some level it’s a better experience.
It’s a more private/personal experience. Turning on the TV means everyone watches.
> It’s common on open floor plans / large rooms for a couch to end up in a completely arbitrary distance from a TV rather than next to a wall. Further setting up the TV on the width vs length vs diagonal of a room commonly provides two or more options for viewing distance.
You’re essentially arguing that people can arrange their furniture for the best viewing experience. Which is true, but also not what people actually do.
The set of people willing to arrange their furniture for the best movie watching experience in their home are the least likely to buy a small TV.
People still do this while home alone, you’re attacking a straw man.
> least likely to buy a small TV.
People can only buy what actually exists. My point was large TV’s “have been out for decades they really aren’t a replacement” people owning them still went to the moves.
> People still do this while home alone, you’re attacking a straw man.
Maybe? You’re making blind assertions with no data. I have no idea how frequently the average person sits in front of their 60” TV by themselves and watches a movie on their tablet. My guess is not very often but again, I have no data on this.
> My point was large TV’s “have been out for decades they really aren’t a replacement” people owning them still went to the moves.
And we come back to the beginning where your assertion is true but also misleading.
Most people have a large tv in their homes today. Most people did not have this two decades ago, despite then being available.
The stats agree. TV sizes have grown significantly.
> Maybe? You’re making blind assertions with no data.
I’ve seen or talked to more than five people doing it (IE called them, showed up at their house, etc) and even more people mentioned doing the same when I asked. That’s plenty of examples to say it’s fairly common behavior even if I can’t give you exact percentages.
Convince vs using the TV remove was mentioned, but if it’s not worth using the remote it’s definitely not worth going to the moves.
I do. I’ve researched the optimal distance for a smallish tv screen (which fits between the studio monitor stand). I move the tv closer when watching a film, it stands on hacked together wooden box like thing which has some yoga tools and film magazines in it - it has wheels. Crazy stuff.
There is a flipchart like drawing of my daughter covering the tv normally which we flip when watching films.
Living rooms are not that big to start with. I don't think you actually asked anyone's opinion on this! :D
Small TVs are not comfortable to watch. No one I know is okay with getting a smaller TV and moving their sofa closer. That sounds ridiculous. If there's any comfort to this capatilistic economy, it is the availability of technology at throw away prices. Most people would rather spend on a TV than save the money.
As for the theatre being obsolete, I do agree with you, atleast to some extent. I think everyone is right here. All factors combined is what makes going to the theatre not worth the effort for most of the movies. It's just another nice thing, not what it used to be.
Also, the generational difference too. I think teen and adolescents have a lot of ways to entertain themselves. The craze for movies isn't the same as it used to be. And we grew old(er). With age, I've grown to be very picky with movies.
HBO is expensive and most people don't have it. Ergo most people never see or hear about their lower quality content. Only the good stuff that their rich friends rave about.
You not recognizing their shows doesnt mean they are bad. Ive seen most of those and the overwhelming majority are at least solid. I understand netflix's business model, Im just annoyed that theyre buying HBO because they will likely make it worse. Maybe netflix wants more prestige content and will let HBO be HBO but I doubt it.
Yeah until Netflix adds tiered pricing for content and you end up paying more than what Netflix + HBO Max together would have cost because Netflix is the only game in town for that content..
I think like all media consolidation this will send a lot of people back to the seven seas..
Honest question: given all the companies and people working on anti-cheat systems for the last 20+ years of multiplayer video games, don't you think it would all be server-side if it could be, by now?
No, game companies are simply unwilling to pay for the talent and man hours that it takes to police their games for cheaters. Even when they are scanning your memory and filesystem they don't catch people running the latest rented cheat software.
Cheating is a social problem, not a technical issue. Just give the community dedicated server possibility (remember how back in the days games used to ship with dedicated server binaries?) and the community can police for free! Wow!
Yes, I would also prefer that servers were community run as in the hl2 days.
I would still argue that there are technical issues leading to some amount of cheating. In extraction shooters like Hunt Showdown, Escape From Tarkov and a few others, people can run pcie devices that rip player location and other information from the machines memory in order to inject it into an overlay with a 2nd computer, and they do go to these lengths to cheat, giving them a huge advantage. It wouldn't be possible to rip that info from memory for these "ESP cheats" if the server didn't needlessly transmit position information for players that aren't actually visible. IMO this is a technical failure. There are other steps that could be taken as well, which just aren't because they're hard.
Yes, because players want to spend time moderating other players instead of playing the game. Sounds fun!
Community servers literally invented anti-cheat. All current big name anti-cheats started as anti-cheats for community servers. And admins would choose to use them. Game developers would see that and integrate it. Quake 3 Arena even added Punkbuster in a patch.
Modern community servers like FiveM for GTAV, or Face-It and ESEA for CS2 have more anti-cheats, not less.
No, because most companies will make decisions based on time/effort/profitability, and because client-side anticheat is stupid simple and cheap, that's what they go with. Why waste their own server resources, when they can waste the user's?
So it is the company prioritising their bottom line at the expense of their customer's computers. More simply, they move cost from their balance sheet and convert it into risk on the customer's end.
Which is actively customer-abusive behavior and customers should treat it with the contempt it deserves. The fact that customers don't, is what enables such abuse.
This is such a weird take. In an online multiplayer game the cheaters are the risk to the company's bottom line.
If a game is rampant with cheaters, honest paying players stop showing up, and less new players sign up. The relatively small percentage of cheaters cost the company tons of sales and revenue.
It is actively in a company's best interest to do everything they possibly can to prevent cheating, so the idea that intentionally building sub-par anti-cheat is about "prioritising their bottom line" seems totally absurd to me.
Not to mention these abstract "the company" positions completely ignore all the passionate people who actually make video games, and how much most of them care about fair play and providing a good experience to their customers. No one hates cheaters more than game developers.
> because most companies will make decisions based on time/effort/profitability, and because client-side anticheat is stupid simple and cheap, that's what they go with. Why waste their own server resources, when they can waste the user's?
And my comment was a response to that statement. In context of that statement, companies are indeed choosing to prioritise their commercial interests in a way that increases the risk to the computers of their customers.
> Not to mention these abstract "the company" positions completely ignore all the passionate people who actually make video games
Irrelevant. Companies and their employees are two different distinct entities and a statement made about one does not automatically implicate the other. Claiming, for example, that Ubisoft enables a consistent culture of sexual harassment does not mean random employees of that company are automatically labeled as harassers.
Coming to anti-cheat, go ahead and fight them all you want. That's not a problem. Demanding the right to introduce a security backdoor into your customer's machines in order to do that, is the problem.
> pretty much entirely just generalizations of their own experience, but phrased as if they're objective truth
I mean you're describing 90% of blog and forum posts on the Internet here.
This (IMO - so it's not ironic) is the biggest leap most people need to make to become more self-aware and to better parse the world around them: recognizing there is rarely an objective truth in most matters, and the idea that "my truth is not your truth, both can be different yet equally valid" (again in most cases, not all cases).
I think my issue is that the blog post comes across to me as in essence an argument that the person communicating shouldn't be dissuaded by potential reactions to what they say, but it fails to account for the difference between good-faith and bad-faith reactions. There's a huge difference between a bad-faith misinterpretation and a good-faith misunderstanding in my opinion, as the latter seems to come just as often from a failure on the part of the communicator to be clear as from any fault on the listener. It's hard for me not to get the impression that the author either can't or doesn't seen the value in differentiating between those cases based on the fact there's such significant room for improvement in clarifying their views in their paragraph about remote work, which is why I called it out.
A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.
The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits.
Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production.
So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.
on top of that Google is also cloud infrastructure provider - contrary to OpenAI that need to have someone like Azure plug those GPUs and host servers.
The own shovels for own mines strategy has a hidden downside: isolation. NVIDIA sells shovels to everyone - OpenAI, Meta, xAI, Microsoft - and gets feedback from the entire market. They see where the industry is heading faster than Google, which is stewing in its own juices. While Google optimizes TPUs for current Google tasks (Gemini, Search), NVIDIA optimizes GPUs for all possible future tasks. In an era of rapid change, the market's hive mind usually beats closed vertical integration.
Selling shovels may still turn out to be the right move: Nvidia got rich off the cryptocurrency bubble, now they're getting even richer off the AI bubble.
Having your own mines only pays off if you actually do strike gold. So far AI undercuts Google's profitable search ads, and loses money for OpenAI.
So when the bubble pops the companies making the shovels (TSMC, NVIDIA) might still have the money they got for their products and some of the ex-AI companies might least be able to sell standard compliant GPUs on the wider market.
And Google will end up with lots of useless super specialized custom hardware.
It seems unlikely that large matrix multipliers will become useless. If nothing else, Google uses AI extensively internally. It already did in ways that weren’t user-visible long before the current AI boom. Also, they can still put AI overviews on search pages regardless of what the stock market does. They’re not as bad as they used to be, and I expect they’ll improve.
Even if TPU’s weren’t all that useful, they still own the data centers and can upgrade equipment, or not. They paid for the hardware out of their large pile of cash, so it’s not debt overhang.
Another issue is loss of revenue. Google cloud revenue is currently 15% of their total, so still not that much. The stock market is counting on it continuing to increase, though.
If the stock market crashes, Google’s stock price will go down too, and that could be a very good time to buy, much like it was in 2008. There’s been a spectacular increase since then, the best investment I ever made. (Repeating that is unlikely, though.)
How could Google's custom hardware become useless? They've used it for their business for years now and will do so for years into the future. It's not like their hardware is LLM specific. Google cannot lose with their vast infrastructure.
Meanwhile OpenAI et al dumping GPUs while everyone else is doing the same will get pennies on the dollar. It's exactly the opposite to what you describe.
I hope that comes to pass, because I'll be ready to scoop up cheap GPUs and servers.
Same way cloud hardware always risks becoming useless. The newer hardware is so much better you can't afford to not upgrade, e.g. an algorithmic improvement that can be run on CUDA devices but not on existing TPUs, which changes the economics of AI.
> And Google will end up with lots of useless super specialized custom hardware.
If it gets to the point where this hardware is useless (I doubt it), yes Google will have it sitting there. But it will have cost Google less to build that hardware than any of the companies who built on Nvidia.
Right, and the inevitable bubble pop will just slow things down for a few years - it's not like those TPUs will suddenly be useless, Google will still have them deployed, it's just that instead of upgrading to a newer TPU they'll stay with the older ones longer. It seems like Google will experience much less repercussions when the bubble pops compared to Nvidia, OpenAI, Anthropic, Oracle etc. as they're largely staying out of the money circles between those companies.
I think people are confusing the bubble popping with AI being over. When the dot-com bubble popped, it's not like internet infrastructure immediately became useless and worthless.
that's actually not all that true... a lot of fiber that had been laid went dark, or was never lit, and was hoarded by telecoms in an intentional supply constrained market in order to drive up the usage cost of what was lit.
If it was hoarded by anyone, then by definition not useless OR worthless. Also, you are currently on the internet if you're reading this, so the point kinda stands.
Google uses TPUs for its internal AI work (training Gemini for example), which surely isn't decreasing in demand or usage as their portfolio and product footprint increases. So I have a feeling they'd be able to put those TPUs to good use?
Deepmind gets to work directly with the TPU team to make custom modifications and designs specifically for deepmind projects. They get to make pickaxes that are made exactly for the mine they are working.
Everyone using Nvidia hardware has a lot of overlap in requirements, but they also all have enough architectural differences that they won't be able to match Google.
OpenAI announced they will be designing their own chips, exactly for this reason, but that also becomes another extremely capital intensive investment for them.
This also doesn't get into that Google also already has S-tier dataceters and datacenter construction/management capabilities.
Isn’t there a suspicion that OpenAI buying custom chips from another Sam Altman venture is just graft? Wasn’t that one of the things that came up when the board tried to out him?
> Deepmind gets to work directly with the TPU team to make custom modifications
You don't think Nvidia has field-service engineers and applications engineers with their big customers? Come on man. There is quite a bit of dialogue between the big players and the chipmaker.
They do, but they need to appease a dozen different teams from a dozen different labs, forcing nvidia to take general approaches and/or dictating approaches and pigeonholing labs into using those methods.
Deepmind can do whatever they want, and get the exact hardware to match it. It's a massive advantage when you can discover a bespoke way of running a filter, and you can get a hardware implementation of it without having to share that with any third parties. If OpenAI takes a new find to Nvidia, everyone else using Nvidia chips gets it too.
This ignores the way it often works: Customer comes to NVDA with a problem and NVDA comes up with a solution. This solution now adds value for every customer.
In your example, if OpenAI makes a massive new find they aren't taking it to NVDA.
Nvidia has the advantage of a broad base of customers that gives it a lot of information on what needs work and it tries to quickly respond to those deficiencies.
Nvidia doesn't have the software stack to do a TPU.
They could make a systolic array TPU and software, perhaps. But it would mean abandoning 18 years of CUDA.
The top post right now is talking about TPU's colossal advantage in scaling & throughput. Ironwood is massively bigger & faster than what Nvidia is shooting for, already. And that's a huge advantage. But imo that is a replicateable win. Throw gobs more at networking and scaling and nvidia could do similar with their architecture.
The architectural win of what TPU is more interesting. Google sort of has a working super powerful Connection Machine CM-1. The systolic array is a lot of (semi-)independent machines that communicate with nearby chips. There's incredible work going on to figure out how to map problems onto these arrays.
Where-as on a GPU, main memory is used to transfer intermediary results. It doesn't really matter who picks up work, there's lots of worklets with equal access time to that bit of main memory. The actual situation is a little more nuanced (even in consumer gpu's there's really multiple different main memories, which creates some locality), but there's much less need for data locality in the GPU, and much much much much tighter needs, the whole premise of the TPU is to exploit data locality. Because sending data to a neighbor is cheap, sending storing and retrieving data from memory is slower and much more energy intense.
CUDA takes advantage of, relies strongly on the GPU's reliance in main memory being (somewhat) globally accessible. There's plenty of workloads folks do in CUDA that would never work on TPU, on these much more specialized data-passing systolic arrays. That's why TPUs are so amazing, because they are much more constrained devices, that require so much more careful workload planning, to get the work to flow across the 2D array of the chip.
Google's work on projects like XLA and IREE is a wonderful & glorious general pursuit of how to map these big crazy machine learning pipelines down onto specific hardware. Nvidia could make their own or join forces here. And perhaps they will. But the CUDA moat would have to be left behind.
But it's still something grafted onto the existing architecture, of many grids with many blocks with many warps, and lots and lots of coordination and passing intermediary results around. It's only a 4x4x4 unit, afaik. There's still a lot of main memory being used to combine data, a lot of orchestration among the different warps and blocks and grids, to get big matrices crunched.
The systolic array is designed to allow much more fire and forget operations. It's inputs are 128 x 128 and each cell is its own compute node basically, shuffling data through and across (but not transitting a far off memory).
TPU architecture has plenty of limitations. It's not great at everything. But if you can design work to flow from cell to neighboring cell, you can crunch very sizable chunks of data with amazing data locality. The efficiency there is unparalleled.
Nvidia would need a radical change of their architecture to get anything like the massive data locality wins a systolic array can do. It would come with massively more constraints too.
It's not that the TPU is better than an NVidia GPU, it's just that it's cheaper since it doesn't have a fat NVidia markup applied, and is also better vertically integrated since it was designed/specified by Google for Google.
TPUs are also cheaper because GPUs need to be more general purpose whereas TPUs are designed with a focus on LLM workloads meaning there's not wasted silicon. Nothing's there that doesn't need to be there. The potential downside would be if a significantly different architecture arises that would be difficult for TPUs to handle and easier for GPUs (given their more general purpose). But even then Google could probably pivot fairly quickly to a different TPU design.
The T in TPU stands for tensor, which in this context is just a fancy matrix. These days both are optimised for matrix algebra, i.e. general ML workloads, not just LLMs.
If LLMs become unfashionable they’ll still be good for other ML tasks like image recognition.
Nothing in principle.
But Huang probably doesn't believe in hyper specializing their chips at this stage because it's unlikely that the compute demands of 2035 are something we can predict today.
For a counterpoint, Jim Keller took Tenstorrent in the opposite direction. Their chips are also very efficient, but even more general purpose than NVIDIA chips.
How is Tenstorrent h/w more general purpose than NVIDIA chips? TT hardware is only good for matmuls and some elementwise operations, and plain sucks for anything else. Their software is abysmal.
Of course there's the general purpose RISC V CPU controller component but also, each NPU is designed in troikas that have one core reading data in, one core performing the actual kernel work, and the third core forwarding data out.
For users buying H200s for AI workloads, the "ASIC" tensor cores deliver the overwhelming bulk of performance. So they already do this, and have been since Volta in 2017.
To put it into perspective, the tensor cores deliver about 2,000 TFLOPs of FP8, and half that for FP16, and this is all tensor FMA/MAC (comprising the bulk of compute for AI workloads). The CUDA cores -- the rest of the GPU -- deliver more in the 70 TFLOP range.
So if data centres are buying nvidia hardware for AI, they already are buying focused TPU chips that almost incidentally have some other hardware that can do some other stuff.
I mean, GPUs still have a lot of non-tensor general uses in the sciences, finance, etc, and TPUs don't touch that, but yes a lot of nvidia GPUs are being sold as a focused TPU-like chip.
Is it the Cuda cores that run the vertex/fragment/etc shaders in normal GPUs? Where does the ray tracing units fit in? How much of a modern Nvidia GPU is general purpose vs specialized to graphics pipelines?
Except the native width of Tensor Cores are about 8-32 (depending on scalar type), whereas the width of TPUs is up to 256. The difference in scale is massive.
That's pretty much what they've been doing incrementally with the data center line of GPUs versus GeForce since 2017. Currently, the data center GPUs now have up to 6 times the performance at matrix math of the GeForce chips and much more memory. Nvidia has managed to stay one tape out away from addressing any competitors so far.
The real challenge is getting the TPU to do more general purpose computation. But that doesn't make for as good a story. And the point about Google arbitrarily raising the prices as soon as they think they have the upper hand is good old fashioned capitalism in action.
For sure, I did not mean to imply they could do it quickly or easily, but I have to assume that internally at Nvidia there's already work happening to figure out "can we make chips that are better for AI and cheaper/easier to make than GPUs?"
> what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.
Nothing prevents them per se, but it would risk cannibalising their highly profitable (IIRC 50% margin) higher end cards.
It’s not binary. It’s not existential. What’s at stake for Nvidia is its HUGE profit margins. 5 years from now, Nvidia could be selling 100x as many chips. But its market cap could be a fraction of what it is now if competition is so intense that its making 5% profit margin instead of 90%.
My personal guess would be what drives the cost and size of these chips is the memory bandwidth and the transcievers required to support it. Since transcievers/memory controllers are on the edge of the chip, you get a certain minimum circumference for a given bandwidth, which determines your min surface area.
It might be even 'free' to fill it with more complicated logic (especially one that allows you write clever algorithms that let you save on bandwidth).
I think the pace of progress and innovation has, for better or worse, meant that companies can no longer count on successfully evolving only from the inside through re-training and promotions over the average employee's entire career arc (let's say 30 years).
The reality is that too many people who seek out jobs in huge companies like IBM are not looking to constantly re-invent themselves and learn new things and keep pushing themselves into new areas every 5-10 years (or less), which is table stakes now for tech companies that want to stay relevant.
reply