Life is not superhero movie and you have to deal with assholes on a daily basis. You can get away with what you're proposing if you already have an escape plan (another workplace). If they're your coworkers, following your first gut instict is bad.