From experience, if you work somewhere with very disconnected higher ups a sales person shows what your token usage is vs spend on the monthly subs for whatever set of users you have and then show an API price with a sample bill for even “more users” at that same number of tokens that is 30% lower. The crux of course being that your existing user base is not going to use fewer toke a just because you signed more people up and have a discount for the status quo.
Then when you consolidate all users to a single provider you find out you have a 2x to 3x larger bill because you… logically have more users at the same average token spend than the quote.
At my previous job, all while managing a full team, I did a lot of porting of legacy apps to modern standards to maintain various obligations. It was AI gen work in between meetings and paperwork. It was seen as such a success that I was asked to move to an IC role to focus on it within the “Elite new products team” that was supposed to be all AI and the latest tech. I showed up to find that they were pushing out low quality manually written code in 12 hour days six days per week to appear like LLMs were in use to save face because they lacked a token budget or the skills to properly manage one under the APIs. They had dismissed someone who had spent multiple thousands per month on Agent swarms. It was the perfect situation to make their 3x budget issue a 10-30x budget spend issue.
For many many reasons I left that job and got a new one but that was the cherry on top.
Let’s separate out what the companies actually want from the words they’re using for a moment. The top commercial companies are roughly stating:
1. Our AIs are so advanced they’re busting out of our networks and hacking people. We can’t seem to stop them.
2. There are real dangers to just how advanced our AIs are advancing.
3. Someone needs to slow us down
If these were true statements then they’d likely want to shut down themselves for the sake of the planet they live on. If there was a real problem as they have stated then shutting down is the solve (and to be clear, I’m not saying there aren’t problems with the AI industry. I just think they’re BSing us).
In reality this is marketing nonsense (better get in before the ban!) and “our” AI is never open weight/Chinese AI models which they have a different lobbying arm against. If the top four companies got a loosely regulated monopoly through regulatory capture, they’d be giddy.
I’m in the US but I’m basically charging the same way. 16A 240v. I’ve accidentally stepped it down to 13A and didn’t really notice. If you sleep, and can plug your car in while you do so, you can charge.
I think you’re experiencing paging to main memory rather than anything ReBAR related. ReBAR itself reduces memory copy operations. It’s of course possible you’re experiencing some sort of BIOS bug.
Resizable Base Address Register. In short, rather than the 256MB of mapped memory available to the CPU for any PCI device, ReBAR capable devices can map larger memory to the CPU’s addressable memory space. Without ReBAR or sufficient tricks (that Intel GPUs lack) you have to shuffle 256MB chunks around between GPU and CPU.
Using an Intel discrete GPU on a system that lacks ReBAR support results in somewhat lower performance, not a complete inability to use more than 256MB of VRAM.
Source? I don't know what happened for the Linux drivers, but the Windows drivers definitely booted and ran benchmarks without ReBAR when the Arc A380 launched, because there were lots of published comparisons of the performance impact on workloads that definitely didn't fit in 256MB.
Unfortunately I can't edit my earlier post, because I can't find it - might have been distorted rumour that reached me, then :/
Wasn't helped by my experience reacting very strongly to story of early Arc drivers being problematic because the devs were too used to GPU being just a ring-bus away ;-)
Mac OS 7 “Virtual Memory” was just the swap file. Fundamentally classic Mac OS had a flat memory layout with special hooks defined that did not allow for virtual memory as more commonly defined. Multiprocessing came in asymmetric and symmetric forms but really wasn’t terribly useful until 8.5/8.6 and later but by that time Mac OS X was about to show up which solved all these issues.
They can just add those people to the board. It’s not a set number and they don’t regularly add or remove people unless they think they can bring something to the table. Sometimes that is simply “holds a bajillion shares” but if they wanted a High Frequency Trading CEO on the board for insight into that, they could add both his kid and that CEO.
I have a fun entry to the exception to the topping rule. I have a diseased sycamore that was really struggling. The disease was primarily at the top of the tree. We were going to cut it down to prevent neighboring trees from being infected or the tree from falling. The cutting company had a “hold my beer” style suggestion that topping it would bring the tree back. Boy did it. It’s flourishing with new branches from the cut and beautiful coverage. The fungus hasn’t come back and I’m not worried about it dying and falling on my house. Everyone wins!
I would in such a scenario expect the GPUs to be dumped to industrial breakers who would send them to China for refurbishment and repackaging before being sold again on Amazon, AliExpress, and Taobao as last gen gaming cards from weird brands and specs.
This is what happened after the great crypto GPU dumping.
The e-waste recyclers are pretty low on the pecking order, as the creditors will be first to strip these places for assets as Leopold Aschenbrenner discovered. =3
It's where the stuff ultimately goes since the creditors aren't interested in GPUs that no longer have much value. The context here is a crash, not just a basic bankruptcy. If it were that then absolutely it would be impounded by creditors (virtually or in reality) and then sold to the highest bidding data center.
>creditors aren't interested in GPUs that no longer have much value
Indeed, there is a point where the cost of disposal is higher than the expected market value of components. Thus, the asset turns into a liability if held too long.
I have seen factory liquidations, and everything goes... right down to the bolts in the floors. =3
Yeah, I ment it as a joke - I agree with you. Watched the Gamers Nexus GPU investigation recently, where they were shown how a chinese soldering shop can transplant GPU chips to a new board, including memory chip reuse.
Hopefully we can look forward to all that useless datacenter AI crap gets repurposed in a similar manner into something actually useful for users.
That anecdotally matches my experience thinking about it.
After my initial hearing damage from someone clearing a rifle before everyone put on hearing protection it took a while for it to attenuate to background noise but flares beyond that volume. Things that certainly avoid flares is when I have a good sleeping setup where I avoid being crumpled or my neck kinked. I have good posture in general but sometimes weird working locations, off site meetings, hackathons, etc would lead to my tinnitus getting worse.
I chalked that up to being physically tired from travel and lacking sleep (both of which seems to also increase my tinnitus)but when reflecting back I don’t recall having much Tinnitus when my son was a newborn through a one year old and I basically slept walk that whole year.
I think on the count back the neck posture and likely the muscles in my neck and head lacking tension is what is keeping it from being worse.
Then when you consolidate all users to a single provider you find out you have a 2x to 3x larger bill because you… logically have more users at the same average token spend than the quote.
At my previous job, all while managing a full team, I did a lot of porting of legacy apps to modern standards to maintain various obligations. It was AI gen work in between meetings and paperwork. It was seen as such a success that I was asked to move to an IC role to focus on it within the “Elite new products team” that was supposed to be all AI and the latest tech. I showed up to find that they were pushing out low quality manually written code in 12 hour days six days per week to appear like LLMs were in use to save face because they lacked a token budget or the skills to properly manage one under the APIs. They had dismissed someone who had spent multiple thousands per month on Agent swarms. It was the perfect situation to make their 3x budget issue a 10-30x budget spend issue.
For many many reasons I left that job and got a new one but that was the cherry on top.
reply