Publication Bias

Imagine a filing cabinet full of studies that found nothing. Nobody will ever read them.

Journals publish results that are new, surprising, and positive. Studies that find "no effect" are technically just as valuable — they tell you what doesn't work — but they're boring. They don't make headlines. They don't advance careers. So they sit in researchers' file drawers, unpublished and invisible. This is the file drawer effect, and it warps everything.

When only positive results see the light of day, the published record becomes a highlight reel, not a representative picture. A drug might fail in nine out of ten trials, but if only the one success gets published, the literature says the drug works. Doctors prescribe based on that literature. Patients take those drugs. Nobody sees the nine failures because nobody published them.

This isn't a minor technicality. It systematically distorts the body of knowledge that researchers, doctors, and policymakers rely on. Meta-analyses — studies of studies — inherit the same bias because they can only analyse what's been published. The foundation looks solid, but there's a hole where all the inconvenient evidence should be.

You can't evaluate a claim by looking at the evidence for it. You have to ask: what evidence might exist that you're not seeing?

---

References

• John Ioannidis — Why Most Published Research Findings Are False (2005)
• Ben Goldacre — Bad Science (2008)