It will likely occur, just maybe not this year or next. If we look over the last eighty years of computing, the trend has been smaller and more powerful computers. No reason to think this won’t occur with running inference on larger models.
We had a lemon tree that did this. The irrigation line connector was probably not 100% sealed and the roots grew to it slowly broke it. It enabled the lemon tree to gets lots of water and grow. Meanwhile the trees further down the irrigation line suffered.
Why wouldn’t 3rd party hardware vendors continue to work on reducing costs of running models locally? If there is a market opportunity for someone to make money, it will be filled. Just because the cloud vendors don’t develop hardware someone will. Apple has vested interest in making hardware to run better models locally, for example.
> Why wouldn’t 3rd party hardware vendors continue to work on reducing costs of running models locally?
Every one wants this to happen they are all trying but...
EUV, what has gotten us down to 3nm and less is HARD. Reduction in chip size has lead to increases in density and lower costs. But now yields are DOWN and the design concessions to make the processes work are hurting costs and performance. There are a lot of hopes and prayers in the 1.8 nodes but things look grim.
Power is a massive problem for everyone. It is a MASSIVE a problem IN the data center and it is a problem for GPU's at home. Considering that locally is a PHONE for most people it's an even bigger problem. With all this power comes cooling issues. The industry is starting to look at all sorts of interesting ways to move heat away from cores... ones that don't involve air.
Design has hit a wall as well. If you look at NVIDIA's latest offering its IPC, (thats Instructions Per Clock cycle) you will find they are flat. The only gains between the latest generation and previous have come from small frequency upticks. These gains came from using "more power!!!", and thats a problem because...
Memory is a problem. There is a reason that the chips for GPU's are soldered on to the boards next to the processors. There is a reason that laptops have them soldered on too. CAMM try's to fix some of this but the results are, to say the least, disappointing thus far.
All of this has been hitting cpu's slowly, but we have also had the luxury of "more cores" to throw at things. If you go back 10-15 years a top end server is about the same as a top end desktop today (core count, single core perf). Because of all of the above issues I don't think you are going to get 700+ core consumer desktops in a decade (current high end for server CPU)... because of power, costs etc.
Unless we see some foundational breakthrough in hardware (it could happen), you wont see the normal generational lift in performance that you have in the past (and I would argue that we already haven't been seeing that). Someone is going to have to make MAJOR investments in the software side, and there is NO MOAT by doing so. Simply put it's a bad investment... and if we can't lower the cost of compute (and it looks like we can't) its going to be hard for small players to get in and innovate.
First time filing on our own in 25 years and it was great experience. Paid for the state filing. They didn’t seem to really push the add ons for federal.
Home Assistant is to HomeKit is very much like AppleScript/scripting languages are to Shortcuts. The UI based approach is OK, but only takes you so far because there is not proper investment to improve a deeper integration. To make the most and easiest management of the system often requires using a more capable system and bridging the two.
They used no networking services either. Now, I open a shared PowerPoint and am stuck waiting for a couple of minutes while it is syncing or doing who knows what. People have no sense of what templates they copy from other documents causing the size of the file and load times to bloat.
I haven’t developed with .NET in a dozen years, let alone since it went cross platform, but I at least know it is capable of being cross platform. It amazes me how many developers I speak to that still assume .NET is Windows only.
I just found my old Flip Camera in bin of old electronics today and its rubber case was a sticky mess. In the bin is a bunch of old hard drives that haven't been touched in a decade as well. I imagine there is nothing on there I need that hasn't been transferred elsewhere, but also just curious if any of them work.
Not sure what time you go to TJ's. In SoCal, I go nearly daily, a benefit of living 50ft from the store, and while stock is currently low, I have only seen it empty once. Again, it could be that when I go at lunchtime, they have the daily delivery available, but by the evening it is gone. Their prices for regular eggs remain <$4/dozen.
