thinking about this more (i'm replying to myself!) -- i guess what the experiments for D/K show is exactly that performance on a test is uncorrelated with your idea of the performance on a test.
yes, it's kind of surprising that, having dropped the "real" results in a vat of acid, our hapless researcher replaces the missing data with random numbers and gets the same result -- but that's only because we didn't expect random numbers to model the outcome.
instead, we would have expected that, towards the bottom of the distribution of test-takers, those folks would rate themselves lower, while towards the top they would rate themselves higher. at the extreme of perfect self-awareness, the line for subjective results would exactly match the line for objectively-scored results.
this is the exact argument that is made in the post linked in the top comment:
> by using random data to argue that the Dunning-Kruger effect is not real, the author is arguing to default to the base assumption. But which base assumption do they make? One even more radical than what’s proposed by Dunning-Kruger. In the author’s world, the Dunning-Kruger study should be interpreted in the reverse direction, claiming that there is at least some self-awareness in the way people self-assess.
yes, it's kind of surprising that, having dropped the "real" results in a vat of acid, our hapless researcher replaces the missing data with random numbers and gets the same result -- but that's only because we didn't expect random numbers to model the outcome.
instead, we would have expected that, towards the bottom of the distribution of test-takers, those folks would rate themselves lower, while towards the top they would rate themselves higher. at the extreme of perfect self-awareness, the line for subjective results would exactly match the line for objectively-scored results.
this is the exact argument that is made in the post linked in the top comment: > by using random data to argue that the Dunning-Kruger effect is not real, the author is arguing to default to the base assumption. But which base assumption do they make? One even more radical than what’s proposed by Dunning-Kruger. In the author’s world, the Dunning-Kruger study should be interpreted in the reverse direction, claiming that there is at least some self-awareness in the way people self-assess.
source: https://andersource.dev/2022/04/19/dk-autocorrelation.html