I was curious if the self assessment is done before or after the test.
Bing chat gave me this wild answer:
> The effect is usually measured by comparing self-assessment with objective performance. For example, participants may take a quiz and estimate their performance afterward, which is then compared to their actual results 1. Therefore, people estimate their ability before the test by Dunning-Kruger.
In the case estimation is done before: If you've had training, like a soup of ingredients, that matches the priorities and biases of the test it would be strange if no measurable effect remained.
If it's done after: You can create trick questions specifically designed to test if someone learned a specific thing. A good test would test for that. If someone didn't learn the specific thing they could give/guess the wrong answer with some confidence.
The design of the test has great influence on how poorly you'll think you've done. I would argue that the superior test is the one designed to fool you. Hans Rosling famously created a multiple choice test with 4 answers per question with average results below 25%.
On a more fascinating note, unskilled means all areas of expertise outside your own.
People who are universally unskilled in all areas are of course more likely to think they are unskilled. In reality these people know little bits about many things.
This in contrast with people who spend all day, every day, for their entire lives pondering topics inside their area of expertise. If you are doing one thing you aren't doing all of the other things.
Wikipedia had hilarious instances of experts contributing to countless articles accidentally ending up on the wrong page. Suddenly they have no patience, think they know everything and act like children. It's funny because you cant just ban valuable contributors.
I would love to see this DK test done with professors furthest removed from the area of expertise.
Bing chat gave me this wild answer:
> The effect is usually measured by comparing self-assessment with objective performance. For example, participants may take a quiz and estimate their performance afterward, which is then compared to their actual results 1. Therefore, people estimate their ability before the test by Dunning-Kruger.
In the case estimation is done before: If you've had training, like a soup of ingredients, that matches the priorities and biases of the test it would be strange if no measurable effect remained.
If it's done after: You can create trick questions specifically designed to test if someone learned a specific thing. A good test would test for that. If someone didn't learn the specific thing they could give/guess the wrong answer with some confidence.
The design of the test has great influence on how poorly you'll think you've done. I would argue that the superior test is the one designed to fool you. Hans Rosling famously created a multiple choice test with 4 answers per question with average results below 25%.
On a more fascinating note, unskilled means all areas of expertise outside your own.
People who are universally unskilled in all areas are of course more likely to think they are unskilled. In reality these people know little bits about many things.
This in contrast with people who spend all day, every day, for their entire lives pondering topics inside their area of expertise. If you are doing one thing you aren't doing all of the other things.
Wikipedia had hilarious instances of experts contributing to countless articles accidentally ending up on the wrong page. Suddenly they have no patience, think they know everything and act like children. It's funny because you cant just ban valuable contributors.
I would love to see this DK test done with professors furthest removed from the area of expertise.