Can you delve more into the details-- why does this occur? Maybe you could ask them, "I hope you don't mind my asking, I struggle to make friends and I'm wondering if I did anything off-putting?"