Matrix multiplication is a nice example, but does it also support sparse matrix-vector multiplication (because this is what is most useful in scientific computations, e.g. iterative solvers)?
SIMD-sparse matrix multiplication is possible but much more complicated. That's why BLAS libraries exist.
As long as you're willing to write the full scope of operations in a SIMD style, I'd bet that you could do it. But sparse is a lot more difficult to do than dense.
I know the basics of sparse matrix multiplication (there are many types: COO, CSR, LIL, etc. etc. Each representation would lead to a subtly different matrix multiplication algorithm). Load-balancing these operations across the GPU would be difficult, but it looks like major BLAS libraries have solved the problem already (ex: cuSPARSE, CUDA's sparse-matrix library for GPU compute)
