How would you propose, in this specific study, they verify what you want? Please be specific regarding what questions you want asked, when they should be asked, and how these questions can't be discarded by saying "well, they could be faking it".
Or the people undertaking these studies (and more importantly, the people peer reviewing and publishing the work) push back on broad conclusions that aren't substantiated by the research.
Having seen the nonsense my partner has to put up with in order to get a post-graduate qualification to practice as a professional psychologist I'm not holding my breath. From where I'm sitting the replication crisis in this field looks like a crisis of incompetence and apathy.
> broad conclusions that aren't substantiated by the research.
But these aren't broad conclusions, they are specific conclusions, and they are substantiated by the research. You're just positing that the results could be entirely fake on a whim.
Designing good studies is hard. That doesn’t mean we should just roll over and be satisfied with bad studies. The same applies to most studies about human diet and nutrition.
That is why I asked for some way to test for what GP complained about, and the only answer I got was "there is no way". Do you have a way to design a study so the complaint "well, the participants could be faking it" isn't valid? I'm legitimately asking, so far all I've gotten was "no way".
Not sure if I replied to you further up, or someone else, but in any scientific study you should be attempting to minimize the impact of the observation method used on the outcome of the data, and when it comes to answering questions truthfully, someone being aware that they are participating in a study does not do that at all. They should be using data from real-world situations in which people did not have an incentive to lie. Problem is, those are hard to find, and most studies are done in order for the researchers to get paid so they can pay rent and buy food.
There is far more incentive to conduct shoddy research in psychology (since as a corrolary to being hard to produce good evidence for, it's hard to produce good evidence against, which non-reproducability is often not treated as), than there is to only proceed with the studies you actually have sufficient known-good data to work with.
We have spent billions of dollars on single technology projects in order to answer questions about physics, like the LHC, because without those we often cannot actually get the data needed to answer certain questions.
Nothing equivalent is done in social psychology. There are no massive, multi-use infrastructures being built to engage with it.
While that is not the fault of social scientists, but of the devaluation of their field by the society around them, it still does mean they often do not have the necessary tools to do their jobs correctly.
> That's not the only option. Why is it the only one you can think of?
It's not the only one I can think of, but no other options have been presented to me. I have been asking for solutions, but nobody has provided any that can't be dismissed on the same grounds.
> I mean, where are you going with "Our only options are to draw the wrong conclusions, or to stop studying it altogether"?
Where did I write anything close to that? I am not saying anything about drawing conclusions, I am literally asking: how would you design a test that measures the proposed problem? Why are you accusing me of advocating for "drawing the wrong conclusions"?
It's incredible how many people want to read my comments as advocating for something they are not. If you feel attacked because I'm asking how we could measure your idea, maybe I'm not the problem, but your idea is?
> It's not the only one I can think of, but no other options have been presented to me.
Well, then, with all due respect, if you already know of other options, why are you asking?
> I'm asking how we could measure your idea
Who cares? Even if you cannot "measure" (whatever that means) the alternatives, the fact still remains that people are drawing conclusions from incorrect and invalid data.
That's the point, really - the field of psychology is bereft of replicable studies.
Observing that the way a particular field performs research results in invalid conclusions doesn't put any obligation on the observer to provide alternatives.
It's enough to point out that a thing is wrong; there's no requirement to also provide the correct answer.
> Well, then, with all due respect, if you already know of other options, why are you asking?
If someone engages in a conversation with me and presents a thought, I would like them to present it fully. If there are obvious flaws or implications, I try to reflect them, both so I can be sure I correctly understood the other party, and to give them a chance to fill in those holes. I do not wish to just inject my own ideas everywhere, I want to understand people as they express themselves. Seems like a pretty normal way to talk to other humans to me.
> Who cares? Even if you cannot "measure" (whatever that means) the alternatives, the fact still remains that people are drawing conclusions from incorrect and invalid data.
And because you are declaring the data incorrect and invalid that means it is so? You will either have to share your research credentials which allow you to judge it as such, or you'll have to answer the obvious questions that come after such a statement (e.g. "how can you prove that?"). It's pretty normal in science that instead of people just blurting out thoughts which are taken as truths, we reflect on ideas and ask questions that help us get to a better understanding divided from our subjective point of view. If the best you can come up with is "well, it's obviously true, duh" then your thought isn't as good as you think.
> It's enough to point out that a thing is wrong
You are wrong. There, I said that you are wrong - does that make you wrong? If not, is there maybe some system by which we can determine whether you're right or wrong? Your response to this question is "no need, I am right".
Not all of psychology, but this type of problem, yes. We just don't have the neurological knowledge required to get such information meaningfully yet. It would be like people in the 1700s trying to study nuclear forces - it's not an absurd endeavor, but they just didn't have the means yet.
