This is a common misunderstanding. There are many different types of context switching and you need the context (heh!) To compare apples to apples.
Kernel to userland, thread to thread, process to process, kernel thread vs user thread, etc. It's conflated because different threading models and different kernel/userland separations exist.
Generally speaking (in linux) threads perform much much faster than switching processes.
This is right, and the other mistake people make is measuring just the context switch, and not the many hundreds of thousands of cycles after the context switch spent repopulating the cache.