Yeah, it's useless to look at time-series changes in this data without accounting for the fact that, e.g., 29% of students in Germany now have a migration background. A huge chunk of German students are growing up with parents who don't speak German at home, of course that's going to cause test scores to decline.
Swedish scores sank for both Swedish and maths in "natives", 2nd gen and 3rd gen immigrants. Scores did not sink in Swedish for 1st gen immigrants. In other words: Everyone is getting worse, but some were already worse than others before.