Thursday, August 14, 2014

It’s crap, OK? Deal with it

Scientists are baffled by the education community and their methods
Failure to Replicate (Inside Higher Ed)

     …[P]sychologists are not the worst offenders when it comes to replication, it turns out. That distinction might belong to education researchers, according to an article published today in the journal Educational Researcher.
. . .
     Only 0.13 percent [i.e., one tenth of one percent] of education articles published in the field’s top 100 journals are replications, write Matthew Makel, a gifted-education research specialist at Duke University, and Jonathan Plucker, a professor of educational psychology and cognitive science at Indiana University….
     Makel and Plucker … found that 221 of 164,589 total articles replicated a previous study….
     What’s more, 48.2 percent of the replications were performed by the same research team that had produced the original study….
More colorful "research"
     Replications are an essential part of validating scientific knowledge. They control for sampling errors and weed out fraud. A replication might show, for instance, that an educational intervention’s effects are less pronounced than a previous study contended.
     So why do so few replications appear in education journals? The article, “Facts Are More Important Than Novelty: Replication in the Education Sciences,” argues that education journals routinely prize studies that yield novel and exciting results over studies that corroborate – or disconfirm – previous findings. Conducting replications, the researchers write, “is largely viewed in the social science research community as lacking prestige, originality, or excitement.”
. . . .
     Makel and Plucker, however, say that replication matters greatly. What’s at stake, they say, is education’s standing as a discipline. Dismissing replication, they write, “indicates a value of novelty over truth … and a serious misunderstanding of both science and creativity.”….
. . . .
     “When I talk to my friends in the natural sciences, they’re just baffled by how this is even a question or a controversy in psychology and education,” Makel said. “Replication is such a normal part of the process for them.”....

     See also EdDreck: the "experts"

No comments:

The return of the curious "Shooting Star" saga

     Back in 2012, I wrote a series of posts about the mysterious sinking, in 1974, of the yacht "Shooting Star," which took th...