This is my experience too. My meetings tend to be presentations of results. I invite the bare minimum of people who are likely to be interested, and like you, end up with a full meeting room plus others connecting online, often all over the world.
I figure, they're consenting adults, they're responsible for managing their time.
A lot of mine at the time were workshops. I find workshops work best when there is an agenda and small teams, then you present to the wider group when ideas are more developed. A lot of the time, when additional people attended they would be seeing too early of a concept or idea and too many people would debate little details. I believe the best productivity is in small teams.
I figure, they're consenting adults, they're responsible for managing their time.