My take as well. The dual-3090 and dual-spark configs have an outsized voice in this conversation. I'd add pipeline parallelism vs tensor parallelism and in the case of TP setting your lanes and speeds per card.
Would love to have multi-gpu support eventually but it's a balance between whats popular and what actually has good quality benchmarks available. Multi-gpu numbers can also vary between setups
Influencers have latched onto the pitch that everyday people can run frontier models on an 8gb GPU while sticking it to the labs. There's a large audience of people who haven't had the hardware to test larger quants to see the difference.
They would be better served with smaller models that can reliably call tools and generate structured outputs without looping or totally hallucinating. These projects exist, but aren't getting amplified.
Most popular would be Qwen 9b or Qwen 35b with experts offloaded to CPU. Even these aren't great, but they can be run at higher quants on lower hardware. We'll see what Qwen releases version 4 in the coming weeks.
The article takes a hand-wavy attitude towards abuse and negligence, with little attempt to counter-balance it with transparent accountability. If we can't address that in good faith, I see no reason to give the other 98% of the article weight.
> The article takes a hand-wavy attitude towards abuse and negligence, with little attempt to counter-balance it with transparent accountability.
What do you mean? It addresses the matter explicitly, making the argument that it's hard to abuse with impunity. Is there a particular point you think it should have addressed that it didn't?
Sure, like in the case of a rear/front plate switch on a stolen car, Flock has every bit of information to identify that this is the case before deploying an officer to perform a pit maneuver on an innocent citizen. But they have no incentive to build that, and the author makes no allowances for this kind of self-imposed accountability. The entire tone is "bad things happen, what can ya do?"
> The entire tone is "bad things happen, what can ya do?"
Well, no, it isn't -- for abuse, its response is "we can catch it after the fact and charge or fire the officers" (which then also provides a deterrent effect). I guess the particular sort of case you're describing doesn't fall under abuse (I guess you'd call it negligence?) and so this might not apply in this case. So maybe you could say it takes a hand-wavy approach to negligence. ("Negligence" doesn't seem like quite the right term here -- I assumed you were talking about negligence or abuse on the part of the police, rather than on the part of Flock.) But it explicitly addresses abuse, which is contrary to your original claim that it doesn't address either!
> Sure, like in the case of a rear/front plate switch on a stolen car, Flock has every bit of information to identify that this is the case before deploying an officer to perform a pit maneuver on an innocent citizen.
OK, and how do you know that they do not in fact flag this case so such incidents can be avoided? I agree with you that if they don't than they should, but you haven't provided any indication that they don't, just asserted it.
> But they have no incentive to build that
Why not? Seems to me like the police would find it helpful. Of course maybe there's some nonobvious reason I'm missing why there's no incentive, but you should say what that reason is rather than just asserting it exists!
> and the author makes no allowances for this kind of self-imposed accountability.
I'm not even entirely sure what you mean by this. Like, flagging things in advance to prevent a needless deployment isn't "accountability", it's just getting things right in the first place. I mean yeah you're right that the article doesn't seem to really discuss "what if Flock just gets things wrong?" But I thought we were talking about abuse -- or I guess also negligence? But I was also assuming there you were talking about negligence on the part of the police, not of Flock. It's confusing if you mash these two together!
So sure -- the article ignores questions of "what if Flock just gets things wrong?" If that's what you meant by "negligence", you could have said that a lot more clearly, but I agree it doesn't discuss that. But it definitely addresses abuse.
Also, sorry, but I gotta say, you're making your argument in a very compressed form that makes it hard to determine exactly what you're arguing (and what is referring to exactly what, etc). I had a hard time here determining exactly what you meant in order to respond to it. Presumably someone who already agrees with you would be able to fill in the details in their head -- but those aren't the people you need to be making your argument to! For the sake of those of us who don't already agree with you, it would be helpful to expand a bit.
While I know it's not apples to apples, the target comparison right now is 2x DGX Sparks. Similar price, 256gb. The conversation has focused on memory bandwidth vs. compute in agentic loops, so for most people the raw numbers will mean less than the "time per task" in coding benchmarks.
This is a great article and bodes well for the M5, but we should expect more like this comparing to other platforms before we truly understand where it fits.
Speed vs task-completion is a new conversation and a great point. Whereas the cost to compute doesn't exist in a vacuum, making mistakes costs less, is easier to maintain with granularity and a whole host of other factors when you own the lab.
Can you share what tools and processes you're using to do this?
I've been using Pi to build custom extensions and wrapping workflows in shell processes to make it more deterministic and enforce certain validations, all guided by Fable. This isn't production work, though, just playing llm factorio at home.
What you want is a bunch of sessions to replay. Something anonymized if it's not yours, and something that's not depending on state.
You replay all your sessions against your harness, and then store all logs all output, everything to a safe place.
Finally use a blind judge to check everything, and score the output.
Then fix your harness, iterate again until better until you are in a point where it's just the model's weakness. If you get to that, use a bigger model.
It's exciting, but that's $9k for a model that's in the ballpark of last year's frontier. For many people that's like 10 years of up-front cost in a landscape that's getting better and cheaper every week.
but compared to the $500k+ racks running the current frontier, it does give perspective.
There's great VST plugins like "Sketch Cassette 2" that will do hiss, flutter, all of that, along with some great presets. That one's less about creating authentic sound profiles, more like creating lo-fi tracks and instruments for music production.
reply