Even if the immune system completely destroys the tumor, I'd be worried about tumor detritus (is that the right word?) floating around in the brain? It might be better than the alternative, but I'd think that's one spot you really don't want cobwebs gathering in.

If the immune system actually works properly, no, not really a huge concern. Leukocytes and macrophages fully consume foreign matter, enclosing it in a phagosome. This then merges with a lysosome, where the matter is broken down into harmless components, although some stays on the surface of the cell to activate other immune cells against that particular brand of gunk.