The confusion may come from the various spelling conventions in the numerous loan words. In many of the counterintuitive cases, you could imagine a more phonetic spelling. The tradition has been to preserve buffet as is, instead of rewriting it as, "buffay".
The distinction is there. English can be used phonetically. We prefer to preserve the heritage of various loan words instead.
Hearing Americans pronounce the French loanword 'niche' as 'nitch' instead of 'neesh' is cringe-inducing.
English pronunciation is just kind of a mess (especially in the US). It is one of the few languages where highly educated mature people are regularly unsure of how to pronounce a word in their own language or where there is no agreed upon 'non-dialect'/standard pronunciation.
Some Americans clearly must do this, but personally, I've never heard this in my life until I saw it on a YouTube video of a British person complaining how Americans pronounce words. Obviously, your experience may vary - it's a big country.
The transatlantic dispute over "aluminum/aluminium" seems minor when you consider how English is used globally. Even within Britain, there are considerable variations.
The one that gets me, as an American is nuclear vs nucular. Both have been in use verbally and written for decades... academics have adopted the former, even if the latter was more common in most early use. And that's just one, pretty recent example.
And as a British speaker of English it amuses me that we say "fillit" steak when Americans (afaik) correctly say "fillay". There are others, but I guess there are more 'correct' British English pronunciations of words with origins like this than there are American English.
>It is one of the few languages where highly educated mature people are regularly unsure of how to pronounce a word in their own language
Which is worse, being unable to correctly pronounce a word (but still being close enough to be understandable) or being completely unable to write a word?
...we all agree that the right pronunciation of "nitch" is "neesh", though, or at least I've never heard a serious argument to the contrary. People just genuinely don't know how to pronounce it because they've only seen it written.
One that still gets me personally is "hyperbole"--I know how it's pronounced but when I read it, I still say "hyper-bowl" in my head more often than not. I don't think I've ever made the mistake while reading out loud to someone yet, but it will likely happen some day and when it does I will feel very stupid.
The distinction is there. English can be used phonetically. We prefer to preserve the heritage of various loan words instead.