Not sure its a standard explanation, but i recall reading a couple of research articles about that topic.

I think it is kind of a paradox which is clearly visible in some progressive Scandinavian countries like Sweden or Denmark.

> he could probably bag a few trillion if he can trick humanity into kicking off the biggest space boondoggle

That makes no sense. You don't pick colonizing Mars as a get rich quick scheme. Also his seemingly genuine obsession has revolutionized space launch with Falcon 9's and internet connections with Starlink.

Not sure how either of those are space boondoggles. Even Starship is making huge progress towards something which could help make a moonbase a reality in the next few years.

Recently it seems that even if you add those conditions the LLMs will tend to ignore them. So you have to repeatedly prompt them. Sometimes string or emphatic language will help them keep it “in mind”.

Glad it's not just me then, it's been driving me slightly batty.

Great now I’m envisioning a rich guy using an A100 as his desktop GPU just to show off. Which begs the question if that’s even possible.

It has no video output.

I believe some cards at least you can make the motherboard display ports the output

This is kind of true back in the day though. Uninformed people would buy Quadro cards because they were the most expensive GPU on Newegg only to realize this thing sucks for gaming.

For these large services it seems that small companies should be allowed to sue them.

Otherwise there’s no incentive for the big providers to care.

Similarly for anti-virus. It’s a PITA when Windows or Mac falsely flag a program as a virus when it’s not in their app stores.

Yes, at least in the US, being a litigious freak gets results.

Weird trick to get unblocked: follow the standard three-email procedure to sender support, then send a fourth email ccing [email protected] telling them to unblock or next step is attorney general.

The thing about a lot of attorney generals is they LOVE to smack down a big corporation like Microsoft for the little guy.

Now its available to many standard patients and for more types of cancers. Thats huge progress.

Nope the US has interests and stakes in this going back decades.

Saudi Arabia was also heavily lobbying the US to attack Iran as well (1).

1: https://www.ndtv.com/world-news/us-israel-attack-iran-iran-i...

Indeed, I'vve already read about an opposition government being organized.

That’s pretty awesome!

Though only 5gig Ethernet? Can’t they do usb-c / thunderbolt 40 Gb/s connections like Macs?

It's sad that NDA fetishist Broadcom has a de-facto monopoly on PCIe fabric switches; notably we would have functional open source drivers for at least simpler topologies for a while now, and could just set up cheap FNN topologies by using (usually NMVe targeted) bifurcation support on hosts to get several x4 ports with only a comparatively cheap retimer out into "mini SAS hd" (the square shaped 4-Lane connectors) or QSFP+ ports; and then have a few meters reach on generic DAC cables from such standards (even Skylake-era SAS ones (nominally 12 GT/s; PCIe4.0 is 16 GT/s) should typically manage PCIe4; that's just under 64 Gbit/s from each link, with typical desktop/gaming systems delivering 3~5 links without complaints next to a dGPU (that one at fewer than full lanes).

> Though only 5gig Ethernet? Can’t they do usb-c / thunderbolt 40 Gb/s connections like Macs?

Does the network speed matter that much when TFA talks about outputting a few tens of tokens per second? Ain't 5 Gbit/s plenty for that? (I understand the need to load the model but that'd be local already right?)

Running inference requires sharing intermediate matrix results between nodes. Faster networking speeds that up.

I read (but cannot find this anymore) that the information sent from layer to layer is minimal. The actual matrix work happens within a layer. They are not doing matrix multiplication over the netwerk (that would be insane latency wise).

The LLM/transformers attention layers require an O(n^2) operation between all tokens, which does require significant bandwidth.

Yes the latency hurts performance, that why it’s only achieving ~8tok/s.

I appreciate them showcasing Framework Computer hardware, but they would have got 40 GB/s network performance if they had chosen the Minisorum MS-S1 Max: https://www.minisforum.com/pages/new-release-ms-s1-max-ryzen...

The FC hardware has room for a PCIe x4 with maybe ConnectX-7 @ 100GbE mode.

I really wonder if AMD is going to keep getting walloped on the interconnect or if they'll start upping what's available to consumers, at some point.

> the people in charge think they can replace people with LLMs

Additionally using it as a pretext to fire lots of workers like Amazon and others seem to have been doing. Some friends mentioned their companies using it as a way to offshore to cheaper locales while getting less bad press.