Which is why I said "the first few years". At some point, he realized people in his apartment building were disappearing, and he began to understand his life was in danger if he pissed Dear Leader off too much.