Reproducibility Project Casts Doubt on Classroom SEL Effectiveness
Social Emotional Learning (SEL), teaching students how to regulate their emotions, problem-solve, and disagree respectfully, has been a goal for teaching the “whole child” since 2003 in California. It is being adopted by the rest of the country through the Common Core standards as we wrote here and EdSource reported here. Often the methods for teaching SEL are more like Cass Sunstein’s Nudge towards desired behavior (collaborative projects, sharing time, classroom jobs, etc.) rather than direct instruction. Meta studies, like the journal Child Development analysis of 213 programs, claim to support the effectiveness of SEL in improving student performance on a number of metrics. But a project started at University of Virginia by Brian Nosek could cast doubt on many of those studies and on the overall conclusion that we can predict future behavior based on the conditioning applied.
The Reproducibility Project began as an effort to answer the question about social research studies, “What proportion of results in their field [psychology] are reliable?” Nosek convinced 270 researchers to donate their time to attempting to reproduce the results of 100 published psychological experiments to see if they could get the same results a second time around. Reproducibility is the gold standard of research. Anyone who remembers the cold fusion breakthrough of 1989 understands the importance of reproducibility. If research results cannot be replicated, when fidelity to original design and methodology is high, then the results of the research cannot be held as reliable.
What the Reproducibility Project found in looking at these experiments, which had run the gauntlet of peer review, revision and publication, was that only one third of them were reproducible. In other words, two out of three experiments in behavioral psychology have a fair chance of being worthless. From the RP report,
“these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings (31) despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes.”
The Weekly Standard wrote a good detailed review of the RP’s findings.
The biggest problem noted in many of the studies was confirmation bias which produces a subconscious incentive to find the hypothesized results. In other words, we tend to believe what we want to believe.
The field of social research has been dominated by people of a particular ideology and so many of their studies reach conclusions that match the world view of those with that ideology. What world view is that? TWS said, “In a survey of the membership of the Society for Personality and Social Psychology, 85 percent of respondents called themselves liberal, 6 percent conservative, 9 percent moderate. Two percent of graduate students and postdocs called themselves conservative. ‘The field is shifting leftward,’ wrote one team of social psychologists (identifying themselves as ‘one liberal, one centrist, two libertarians, two who reject characterization,’ and no conservatives). ‘And there are hardly any conservative students in the pipeline.’”
So what we see in social science and human behavioral studies is the confirmation of a liberal viewpoint. TWS wrote, “When researchers, journal editors, peer-review panels, colleagues, and popular journalists share the same beliefs, confirmation bias will flourish. It’s human nature! Reading a typical behavioral science study involving race or sex, privilege or wealth or power, you can find it hard to distinguish between the experimenters’ premises and their conclusions.”
TWS recalled the famous Stanley Milgram studies where he attempted to prove that nearly anyone “could be induced to override his conscience and perform evil acts if he were instructed to do so by a sufficiently powerful authority.” Every student of psychology reads about this study and generally is led to believe that its conclusion, that 65% of people will inflict tremendous pain on others if ordered to do so by a person in authority, is valid. Rarely do such classes consider the study’s reproducibility. Though the overriding assumption has been tested numerous other times, the exact conditions of the original experiment have not been reproduced. Even Milgram’s original findings were reported inaccurately.
Psychologist Gina Perry looked at the original Milgram data, and noted that, of the 600 participants Milgram claimed were in the study, the 65% figure came from only one baseline study of 40 students, all male, college age, paid participants. That means that only 26 students were willing to inflict pain. When the study was replicated, the majority of participants would not go along with inflicting pain.
But the impact of Milgram’s work left an indelible idea about human social psychology in place. “Human beings are essentially mindless creatures at the mercy of internal impulses and outside influences of which they’re unaware. We may think we know what we’re doing most of the time, that we obey our consciences more often than not, that we can usually decide to do one thing and not another according to our own will.”
If that is your baseline understanding of human behavior, how easy is it to promote programs for social emotional learning as a means to make students do what we want them to do, without them even knowing that we are training them to react in predictable, desirable ways?
SEL is incorporated into the Common Core standards and supported by a meta study of 213 psychological experiments. Meta studies assume that the conclusions of the studies they examine are valid, that the null hypothesis has been proved. They do not attempt to recreate the experimental conditions to verify the reproducibility of the results. According to the work of the RP, only 71 of those experiments are likely to have reproducible results, and until someone tries to replicate them, we won’t know which ones; the ones with strong correlation, weak correlation or ones that disproved the null hypothesis.
Combine the goal of teaching the whole child, teaching “life lessons” not just academics, with the concept that humans are mindless creatures who can be nudged into adopting certain behaviors given instruction to do so by people in authority and you have the modern classroom which not only feels justified in directing attitudes, values and behavior, but also believes it will be successful in doing so for ≥65% of the students. Now add in that we have tests that claim to accurately measure those metrics (with no reliability/reproducibility data) and that we will either get rid of or retrain teachers whose students do not demonstrate the desired beliefs and values on these tests and you begin to see the hopeless idiocy of the modern classroom. What we have, as TWS pointed out, is a culture of educrats and media who collude to convince us that these practices will be successful at making a better society for tomorrow. They know, because the studies tell them so. Or not.