The Pashler-Roberts Law: Expense versus Honesty
Tuesday, June 23rd, 2009In this post Andrew Gelman comments on my recent post about acne self-experimentation. He makes an excellent point about drug-company studies:
How would you want to evaluate the risks and effectiveness of a new drug that was developed by a pharmaceutical company at the cost of millions of dollars? I’d be suspicious of an observational study: even if conducted by professionals, there just seem to be too many ways for things to be biased.
Right. And it’s not just observational studies. The data from any big study can be analyzed many ways. The more at stake, the greater the chance of what Andrew calls bias and I call making choices that favor the result you prefer. Independently of Andrew, Hal Pashler and I came up with what I call the Pashler-Roberts Law: The more expensive the research, the less likely the researchers will be honest about it.
You may remember that Robert Gallo, the AIDS researcher, did very expensive research. The deception (possibly self-deception) that accompanied very expensive fusion research is described in Charles Seife’s Sun in a Bottle: The Strange History of Fusion and the Science of Wishful Thinking (2008).
As Andrew says, this is a big virtue of self-experimentation. Because it’s free, it’s easy to be honest, especially about failure. The cheaper the better is a broad truth about science that’s hard to learn from books or classes or even talking to scientists.







