A new paper argues that research into education suffers from a severe flaw: Virtually none of it is ever replicated.
The paper, “Facts Are More Important Than Novelty,” tracks the entire publication history of the top 100 education journals and finds that barely one in a thousand published articles seek to replicate earlier research. The discovery amounts to a crisis for the field, the authors say.
“We cannot know with sufficient confidence that an intervention works or that an effect exists until it has been directly replicated, preferably by independent researchers,” authors Matthew Makel of Duke and Jonathan Plucker of the University of Connecticut write in the paper.
Replication work is considered a crucial part of the scientific process, as it verifies past findings and can discover fraud or errors in methodology that underlied previous breakthrough discoveries. However, replication work is disincentivized by a host of biases. Researchers typically prefer to undertake new research, because brand new findings benefit one’s career more than the confirmation of old ones. In addition, the editors of major journals have a bias in favor of new research over replication work. New research also has an easier time obtaining funding.
Out of over 164,000 education studies observed by the paper, a scant 221 were replication efforts, the paper found. By way of comparison, in psychology about 1 percent of articles are replication, and that field has recently struggled with controversies due to insufficient replication that has allowed flawed studies to go unquestioned. Even worse, of that tiny cohort of replications, only 28 percent sought to directly replicate the original experiment (which is the only way to directly retest earlier research) and nearly half were performed by the same research team as the original study, which means there was no check on possible fraud and higher vulnerability to repeating errors or biases that may have existed in earlier research.
The shortage of replication work done by new research teams is a telling flaw. Overall, 68 percent of replicating studies were completely successful in repeating earlier results, but this rate fell to just 54 percent when a completely new research team was involved.
“If education research wants to be relied upon in the real world, conducting independent replication is essential,” Makel said in a statement accompanying the paper’s publication.
To remedy the situation, the authors suggest that journals could create a new kind of research article, pre-planned “registered replication reports” that would encourage multiple researchers to collaborate in recreating research with a guarantee of later publication. They also suggest that journals could dedicate a portion of every edition exclusively to replication research, to ensure that more of it is published.
There was no word in the authors’ statement about whether a follow-up study was planned to replicate their own findings regarding the lack of replication.
Content created by The Daily Caller News Foundation is available without charge to any eligible news publisher that can provide a large audience. For licensing opportunities of our original content, please contact licensing@
All content created by the Daily Caller News Foundation, an independent and nonpartisan newswire service, is available without charge to any legitimate news publisher that can provide a large audience. All republished articles must include our logo, our reporter’s byline and their DCNF affiliation. For any questions about our guidelines or partnering with us, please contact licensing@dailycallernewsfoundation.org.