Detriment is wholly subjective here. When it comes to politics or social issues, people have different views on how society "should be", and those differences are offensive and considered toxic to the other side. In that sense, everybody is toxic and the label is meaningless and unactionable.
Obviously, most people hold beliefs of some form, and those beliefs will clash with others. Bandying around "toxic" as an absolute in those situations is narrow-minded and prejudiced. Reasonable people with contributions in one neutral area and one controversial area are lumped with this same label as trolls, doxers, and inciters of violence.
Specificity of labels is very important, especially when they are tied to consequences.
Where "toxic" can certainly be used is in relation to a certain goal, like reasonable level-headed discussion, or subscription to a certain worldview; instead of as absolute. Even actions like deliberate trolling aren't considered toxic on sites like 4chan, because that's part of their accepted and promoted culture.
I think the point is, much like offensiveness toxicity is something that is percieved and not objectively determined. Some users and groups may be perceived to be toxic, while much of the user base may consider it positive. Two examples would be the_Donald and /r/late_stage_capitalism. Lot of people think those subreddits are toxic, but lots of users also like and participate in those subreddits.
Safe spaces tend to turn into echo chambers. By reddit banning everything, they've effectively dictated the narrative that's currently found on the site.
