If I continued with the theme, colonisers were poor as fk. who moved to other poor as fk places. But with sufficient luck and skills they were able to enrich their legacies.

Colonisers build colonies in places that had no prior permanent settlement. Thats why we speak of a potential Mars-colony, for the beginning at least.

It seems the wording coloniser is chosen as a power projection, since rich people hold more power than poor people. But it doesn't pay any respect to our past.

Historically, colonization included two very different types of societies- in America, it was primarily poor European peasants moving to undeveloped land and starting farms they worked themselves and pursued an independent future from their former country.

But it was also things like Belgian Congo where a relatively tiny number of Europeans set up governmental & social structures which were exploitative of the local population and extractive of the resources to go back to the colonizing country.

It's unfortunate that we conflate two very different things with one word. Most of the colonists/pilgrims/pioneers/settlers going from Europe->USA were essentially refugees fleeing poverty, persecution, war or starvation in their home countries who were making a new permanent life for themselves. This is in stark contrast to most of the colonization of Africa (whether from Europeans or Arabs) which was primarily military and governmental people who retained allegiance and connection to their home country and often intended to return home after serving some time in the extractive colony.

And of course, in modern usage, it gets used for everything ranging from white rappers to muslim communities in London, so you need a lot of context to assess the intended meaning.