They would not disagree with you after, say, a measles infections, which thoroughly destroys the immune system at many levels.

It’s not a good idea to confuse the impacts of vaccines and infections. Vaccines are carefully developed specifically to help the immune system. Viruses evolve to hurt it.