I have one of my own. My reason for this is that you can get up to 2x performance compared to llama.cpp on the same model weights. It is optimized for single model to serve it well.

I did, the sd.cpp take too much time on my laptop to generate image, the issue is known and it did not seem like it gonna be resolved, so I build my own, and that's why https://github.com/netdur/qwen-image-cplus

Have you considered contributing upstream?

llama.cpp has strict policies on AI-assisted contributions. Most of these specialization projects are done by coding agents. So llama.cpp won't take them (which is fair).

No, it doesn't. AI contributions are welcome if you understand the code. Ping me directly if you want to have something upstreamed.

> if you understand the code

Well, I don't mean to sound rude or anything but that can be a lot to expect from maintainers of those projects.

As I said, it's better for projects like llama.cpp to be selective in this regard.

Well even that is okay. I can use my agent to extract something useful from it.

llama.cpp is a generalized inference engine; therefore, model-specific optimizations should remain outside its core architecture. When developing a bespoke inference application, it is standard practice to fuse operations and heavily optimize the compute kernels for the specific target model.

I have green card and last time I left is 2 years ago, How can I return back to the US?

If you have been outside the U.S. for 2 years without having obtained a reentry permit, then you could be considered to have abandoned your green card. But all isn't necessarily lost. The ability to keep your green card will depend largely on the reason that you were outside the U.S. for 2 years and likely will require you to apply for a returning resident visa.

Where did you see the matmul acceleration support? I couldn't find this detail online.

Apple calls it "Neural Accelerators". It's all over their A19 marketing.

What a ridiculous way to market "linear algebra transistor array".

Hey man, it helps you think different. You just never knew your neurons needed accelerating.

I accelerate them every morning with an Americano.

I have to ask out of curiosity, why is your first comment made with one account, and the reply with a similarly-named alt?

To confuse all those neural accelerators scraping this conversation.

That seems incredibly prescient for accounts created before even GPT-1. Obviously broad data scraping existed before then, but even amongst this crowd I find it hard to believe that’s the real motivator.

Account on laptop, account on mobile.

I really hope someone got fired for this blunder

Which means what, exactly, to someone whose not a machine learning researcher?

Don’t all of the M series chips contain neural cores?

Yes, they do. They're called Neural Engine, aka NPUs. They aren't being used for local LLMs on Macs because they are optimized for power efficiency running much smaller AI models.

Meanwhile, the GPU is powerful enough for LLMs but has been lacking matrix multiplication acceleration. This changes that.

The neural engine is used for the built-in LLM that does text summaries etc., just not third party LLMs.

And there's an official port of Stable Diffusion to it: https://github.com/apple/ml-stable-diffusion

I thought 1 of the reason we do ML on GPU is fast Matrix multiplication ?

So the new engine is accelerator for matmul accelerator ?

From a compute perspective, GPUs are mostly about fast vector arithmetic, with which you can implement decently fast matrix multiplication. But starting with NVIDIA's Volta architecture at the end of 2017, GPUs have been gaining dedicated hardware units for matrix multiplication. The main purpose of augmenting GPU architectures with matrix multiplication hardware is for machine learning. They aren't directly useful for 3D graphics rendering, but their inclusion in consumer GPUs has been justified by adding ML-based post-processing and upscaling like NVIDIA's various iterations of DLSS.

These are different these are built into the GPU Cores

Does this mean that equivalent logic for what has been called Neural Engine is now integrated into each CPU core?

Each GPU core, but yes, this was part of what they announced today - it’s now integral rather than separate.

You can't build to mobile on 4.x version, only desktop supported.

Solar has the advantage of scaling down. 1000 people can give 1M US dollar which can produce approximately 1GW of power on a thousand of power plant in a year. You can scale down as low as 400W of energy production and distribute the financial cost to many people.

As a person from Mediterranean, It finally makes sense the 5% of Nordic genes in my 23andMe result.

I am wondering what is the use case for a website that sends notification to the user who didn't install the PWA to home screen.

Gmail initially offered very big email storage space comparing the others, and it was going up continuously until its storage area merged with Google Drive.

so basically what made gmail amazing was removed in favor of selling you google one storage subscriptions

It depends on the company size. On a small startup with 10 people and a company with 1000 people have different CTO definitions and roles.

Yes, certainly. In a tiny startup the CTO is just a lead developer with more stock options than the others.