I did a huge amount of travel at various times and never had qualms about checking email or doing some other tasks optionally on days off from time to time, but never considered myself to be "working" on weekends.
I've never checked an email on the weekend or in the evening in my nearly 20 years at this current employer, and have never had any qualms about enjoying my time off.
I check it on my own time, but for my own personal curiosity. “I wonder how that team’s dealing with such and such”, that sort of thing. And my job’s inherently on-call 24/7; if something goes badly, I’ll get notified even if I’m not the primary responder, but that’s so rare that I never consider that when making plans.
I’m perfectly content turning my phone off to go camping or whatever. I also don’t feel bad seeing what my coworkers are up to when I’m out of office.
A lot of people are very binary. Yes, I'd internally grumble a bit if a business trip cut into a weekend for travel. I'd also sometimes check email over a weekend or PTO and resolve something if I could quickly. But I certainly didn't go into the office over weekends (or, indeed, latterly at all). And I never took time off for a doctor's appointment or other personal matters taking less than a full day.
And, to your point, if I did take PTO for vacation, the degree to which I'd be contactable or not was never a factor.
I helped a colleague working on a POC for a customer over the weekend. Was fun! Normally I wouldn’t work over the weekend but I find mentoring and teaching colleagues to be rewarding.