Over the past while, a piece of psychology “news” suddenly blew up all over my WeChat feed: the supposedly most classic psychology experiment of them all—the Stanford prison experiment conducted by Zimbardo—was exposed as fabricated. Guokr translated a report from foreign media. According to the report, the participants’ behavior in the Stanford prison experiment may have been deliberately steered by the experimenters, while some participants’ performances were nothing more than “acting” to comply with the researchers’ demands. So the participants’ behavior in the Stanford prison experiment is nowhere near enough to support the hugely influential conclusion that “people’s behavior can change dramatically because of their social roles.”
Well, it looks as if yet another classic psychology experiment has been overturned.
Or perhaps I should put it this way: how many classic psychology experiments are still left that haven’t been overturned?
In just the past two or three years, at least half of psychology’s “cornerstone”-level classic studies may have crumbled:
First and most famously came the OSC’s “massacre”: in 2016, the Open Science Collaboration (OSC) conducted replication tests on 100 studies published in top psychology journals. The result? Fewer than four in ten could be replicated. (The previous post in my Fan Tuan series happened to mention this.)
Next came a whole pile of super-classic experiments getting publicly smacked in the face—
The radish experiment: two meta-analyses, published in 2010 and 2014, concluded that the claim “self-control gets used up” doesn’t hold. Classic experiments—including the “radish experiment”—that supposedly showed willpower being depleted through use were all swept away in one go.
The obedience experiment: in March this year, New Scientist published an article pointing out that the implementation of one of history’s most famous psychology experiments—Milgram’s obedience experiment—was riddled with serious breaches of academic standards. The entire experiment was full of holes. Milgram may even have selectively deleted and concealed results that didn’t fit his expectations. This experiment simply cannot support the conclusion that “people will readily obey orders and commit immoral acts.”
The marshmallow experiment: a paper published earlier this year took down the famous “marshmallow experiment.” It found that, once factors such as family background were controlled for, whether children refused a marshmallow in childhood had nothing to do with their achievements as adults.
After reading all that, aren’t you going to ask: half of psychology’s foundations have been hollowed out—shouldn’t this mansion of psychology come crashing down?
The answer is: this mansion of psychology is still standing perfectly steadily!
Why do I say that?
Because the news about this pile of classic experiments getting smacked in the face should be understood on several levels:
1. Psychology is too young
If you’ve read social psychologist Elliot Aronson’s Not by Chance Alone, you’ll have some idea what I mean: roll the clock back just half a century, and even the world’s top psychologists hardly knew how to conduct scientific experiments!
When Aronson’s mentor Leon Festinger fully articulated cognitive dissonance theory, he didn’t know—or simply hadn’t thought about—how to prove it. The classic experiment on cognitive dissonance theory was eventually completed with Aronson’s help: the student coaching the mentor.
In other words, back when many of these so-called classic experiments were conducted, the psychological-experiment paradigm of tightly controlling variables was nowhere near an academic consensus. Milgram’s obedience experiment, for example, was carried out against roughly this backdrop. Honestly, any properly trained graduate student in experimental psychology today, if transported back to that time, would have enough methodological knowledge to be Milgram’s teacher.
Of course, this is just my personal impression, but Milgram seemed to love the limelight. Given his personality, it’s entirely possible that he deliberately steered participants or intentionally concealed results that didn’t match his expectations. In other words, the possibility that the obedience experiment was deliberately faked is fairly high. And as academic standards develop, this kind of misconduct will also be curbed more and more effectively.
In a word, psychology is too young: the earlier the experiment—even if “early” doesn’t mean all that long ago—the more holes it tends to have. (But isn’t that good news for psychologists today? Our predecessors left holes everywhere, which means plenty of opportunities for us!)
2. Psychology is too difficult
Of course, even an experiment conducted in recent years, with variables strictly controlled, procedures carefully followed, and no fraud involved, still has a very good chance of failing to replicate. Ultimately, that’s because the factors influencing human behavior are too complicated, and human nature is too subtle. When researchers repeat an earlier experiment, some hard-to-detect factors will inevitably shift, interfering with the results.
Besides, the more complex and subtle the psychological process, the more ways it can be disrupted. If all you’re measuring is a simple variable such as reaction speed, chances are it can usually be replicated. But look at the experiments that have been smacked in the face in recent years: they studied willpower, delayed gratification, social roles… all extremely complex and subtle psychological variables. Any unexpected factor could affect how participants behaved in the experiment.
Designing a psychology experiment that is guaranteed to replicate is simply too difficult.
This may well be psychology’s fate: an individual psychology experiment may never produce a conclusion guaranteed not to be overturned. Psychological experiments may only be able to win by ganging up—a whole crowd of experts using different martial arts to hit the same target. The radish experiment has been rejected, but someone may conduct an “apple experiment,” while someone else conducts a “banana experiment.” In the end, perhaps all of them will work. As more and more fists hit the target, the existence of a particular psychological effect will gradually become established.
3. Who did the smacking?
When it comes to being smacked in the face, the third level is to ask: who did the smacking?
If everyone inside a field gets along perfectly and practitioners all say, “We possess unshakable truth,” while outsiders keep challenging it, then that’s when there’s a real problem. It means the ecosystem of the entire field, industry, or discipline is unhealthy. (I’m talking about you, Hongmao Medicinal Liquor and XX.)
But in this latest psychology storm, the people doing the smacking weren’t outsiders. They were professional researchers within psychology. Replacing and overturning the field’s old, less rigorous data and conclusions with more rigorous experiments, data, and conclusions—isn’t that incredibly healthy?
4. After getting smacked, then what?
The fourth level: now that some classic experiments have been overturned, what should we do?
The experiments aren’t reliable, so should we just stop doing experiments altogether and return psychology to Freud’s speculative era?
Of course not. Naturally, our choice is to keep charging forward with swollen faces. Our ambition is to make the next generation of classic psychology experiments less easy to smack in the face.
Comrades, the responsibility is enormous!
References:
1. Guokr. Major expose: Is the Stanford prison experiment an authoritative case study or a lie?
2. Guokr.The marshmallow experiment fails to replicate again! Is not eating candy entirely a matter of willpower?
3. Guokr. Psychology loses another battle: Does self-control really get used up?
4. Dedao column Ma Xujun’s Rapid Reading of the World’s Famous Journals. New Scientist: To what extent would you obey authority?
Note
- This article in the “New Psychology Knowledge Mini-Course” series was a short popular-science essay/ramble written to accompany content related to Zhichao Wei: New Psychology Knowledge Course. It was published on the now-closed platform “Fan Tuan,” and the original link no longer exists.
Leave a Reply