This is an area which I've personally experienced, having spent the first half of my school years in Europe, and being a few days shy of the cut-off point for being held back another year, I was always the youngest in my class (I guess on a large enough population, it would mean that similar kids are the smallest as well, on average).

Later on I switched to a southern-hemisphere school, which was out of phase by 6 months, and I was all of a sudden roughly average age in class. Not that it had a huge impact on me, but it was noticeable, which probably means it has some effect on average.