I find it really strange the way “peer-reviewed” is used by the general public as some gold standard of truth. As a former academic who has been there, the process is extremely arbitrary and variable. Are people aware that the “peer” refers not to a community or a committee, but literally to one random guy or maybe a couple with zero accreditation? And that the journal editor can do whatever they want with this peer’s “review” including completely ignoring it?
Admittedly, peer review originally referred to the fact that the journal being published was read ("reviewed") by your peers, and thus there would be the opportunity for them to provide rebuttals after publication. The process the term now refers to in academia only emerged in the 50's and 60's (Nature for example wasn't "peer reviewed" until 1967). It was literally a marketing ploy to make journals sound more prestigious which got legitimized when certain grant agencies and regulatory bodies started including it in their requirements. All the best science is published in journals that describe themselves as peer reviewed, the government says science not published in such journals isn't up to snuff, and taken literally it seems useful. It's no wonder the public thinks it's an important and long established part of the scientific process.
"I find it really strange the way “peer-reviewed” is used by the general public as some gold standard of truth."
OK but this member of the general public has at least subscribed to New Scientist since 1987, nine O levels, two A levels, two AS levels and a HND in Civ Eng. All pretty mediocre but I have a fair idea on how sciencing is supposed to work and how it ... actually works. Obviously, I ended up in IT.
I should also point out that maths "peer reviewed" is a bit special. For example Mr Wiles went through quite a maelstrom before his proof of some dodgy marginalia was accepted as "true".
The biggest problem is that they are racist. Optical blood sugar sensing devices were developed a decade ago. They were pretty accurate for pale white people, ballpark for most skin tones and don’t work at all for Black people. Good luck bringing that to market. Apple threw out the whole product and restarted with infrared.
Controlled experiments in humans are expensive, time consuming, and actually very difficult to do. Meanwhile any grad student can do a mouse model. The motivation of most academic labs is citations and grants, not useful information. Putting it together…
Selection effects are ridiculously strong for almost anything related to health.
I strongly believe that's why nutrition science is soooo far behind the rest of medicine. There aren't nearly enough serious RCTs (whereas regulations make them abundant for other medical interventions).
