Refcounting is significantly slower under most circumstances. You are literally putting a bunch of atomic increments/decrements into your code (if you can't prove that the given object is only used from a single thread) which are crazy expensive operations on modern CPUs, evicting caches.

Under most circumstances function local variables aren't passed to other threads, or passed at all.

And? That's a small, optional optimization done by e.g. Swift.

Also, I don't know how it's relevant to Go which uses a tracing GC.

It's not "small" if it accounts for most of the allocations :)