Thanks to Oskar Pearson.
My Frontiers of Psychology class read Daryl Bem’s new paper Feeling the Future that reports nine experiments that show an effect of the future on the present. I have a different take than anything I’ve read: I think there are several good reasons to take it seriously. But in this post let’s start with how it could have been better:
1. Lack of background. There have been lots of experiments along these lines. What did they show? This question is not clearly answered. The prior probability of these claims is enormously important. As I told my students, if seeing the future was common and easy for even a small fraction of people, we wouldn’t have businesses, such as casinos, making money on gambling. But the existence of such businesses doesn’t rule out weak effects.
2. Lack of exact repetition. An obvious criticism is that Bem slanted the data analysis to favor the results he wanted. In any data analysis of unfamiliar data, you must choose — how to transform the data, what test to use, and so on. You must also choose how many subjects to run and how many trials to give them. There are rules for these choices (Bem doesn’t seem to know how to choose a transformation) but nevertheless they allow favoritism to creep in. Drug trials have big problems along these lines — severe slanting of the analysis to make the results more favorable — which is why when you register a clinical trial you must specify the endpoints. The answer to the criticism that your data-analysis choices made your favored result more likely is to do a data analysis with no choices at all. This cannot be done from scratch. You need to do the experiment once, make all the necessary choices, and then do the same experiment again (same everything as much as possible) and analyze the data exactly the way you analyzed the data from the first experiment. Bem never does this. Instead each experiment is different from all the rest. This is what experimental psychologists traditionally do but here it is a bad idea. Better to have taken the two simplest and clearest effects (priming and word learning) and repeated them several times exactly.
3. Were experiments left out? Let’s say you observe a weakly-significant result, p = 0.03. Now you do the same experiment eight more times. How likely is it that each of the eight replications will also find a significant difference? Quite low. Yet Bem finds a weakly significant difference in each of his nine experiments. This is highly unlikely. Bem appears unaware of the problem. Mendel had the same problem (data too good to be true). Ultimately Mendel was proved right. But again it stresses that Bem should do exact repetitions and report the results no matter what if he wants to be more persuasive.
Todd Becker pointed me to this post which is negative about the notion that cold showers raise mood (“empty science”) but you can ignore the negativity and go to the comment that gives a long list of studies that support the idea. Todd has blogged about his use of cold showers.
Todd calls this hormesis. About the mood-raising aspect of cold showers, I’m not so sure. There is a broad correlation between being sleepy and being in a bad mood. So anything that wakes us up is likely to improve our mood. But if cold showers improve one’s response to stress of all sorts — which is less clear — it does seem like hormesis in other contexts. When I think of hormesis I think of two sorts: intra-cellular (e.g., x-ray-like radiation breaks stuff, activating repair systems — radiation hormesis) and extra-cellular (microbes in fermented food activate the immune system). But there are other examples of similar stuff: exercise breaks muscle fibers (which is why you shouldn’t exercise the same muscles two days in a row) and longer-term increases them; bones when broken grow back stronger. If we need a certain amount of thermal or other stress to properly respond to stress that would be another example.
This story from the Globe and Mail describes what happened to ten Canadians who left the country to get liberation therapy for their multiple sclerosis (MS). The therapy consists of widening veins that drain blood from the brain. The therapy does not always work, but it usually does. The improvement is so fast and large — comparable to giving someone with scurvy Vitamin C — that the thing being changed must be the source of the problem.
Mainstream MS researchers missed this completely. The mainstream view is that MS is an auto-immune disease (e.g., according to Mayo Clinic staff). This view would never lead you to the liberation surgery. Doctors not only have the wrong idea, they are unwilling to defend it. A woman in the Globe and Mail story tried to get the anti-liberation argument from neurologists. She couldn’t:
Unfortunately the neurologists are all hysterical. You can’t talk to them.
Remember this the next time someone tells you that ulcers are not caused by stress but are actually caused by bacteria — as several contributors to this EDGE symposium claim.
The vast improvement in understanding of MS came about because someone with the necessary expertise (a professor of surgery) cared more than most MS researchers because his wife had MS. I think this is why my self-experimentation found such different solutions than mainstream science: because (a) I cared more than the professional researchers who studied the subject (e.g., sleep) and (b) I had the necessary expertise to do research. I discuss this here.
Thanks to Anne Weiss.
… is the title of a New York Times Magazine article about the ketogenic diet, a treatment for childhood epilepsy, which I’ve blogged about several times (here, here, here, here, here). It’s a very-high-fat diet. It interests me for two reasons: (a) It connects a high-fat diet with proper brain function, as my self-experiments have done. A curious feature of the ketogenic diet is that it isn’t permanent. After several years the child can go off it. My self-experimentation suggests that Americans eat far too little of certain fats. Perhaps eating enough of these fats would prevent childhood epilepsy. (b) It shows how someone who cares enough — in this case, Jim Abrahams, whose son had epilepsy — can be more effective than professional researchers and doctors. Abrahams rediscovered the diet. He saw its value, the professionals didn’t. I’ve argued that this is part of why my self-experimentation found new solutions to common problems: because I had those problems. I cared more about finding a workable solution than researchers in those areas, who had several other concerns (publication, funding, acceptance, etc.).
The details of the article reminded me of something I learned in the BBC series The Story of Science. For hundreds of years, medical students were told, following Aristotle, that the liver has three lobes. It doesn’t. You might think that examination of thousands of actual livers would have dispelled the wrong idea, but it didn’t. The article contains many examples of doctors ignoring perfectly good evidence in favor of nonsense they read in a book or heard in a lecture. Epilepsy is easy to measure. If a child has 100 seizures per day, and has been having them at this rate for years, and this goes down to 5 shortly after he starts the ketogenic diet, and goes up again when the child goes off the diet, there is no doubt the diet works. As early as the 1930s, this had been observed hundreds of times. This was overwhelming evidence of effectiveness. Doctors ignored it, probably based on the modern equivalent of the three-lobed liver. They complained, according to the article, that there was “no evidence it worked” or that the evidence wasn’t “controlled” or “scientific” (whatever that means). A study published in 2008 “answered doubts about keto’s clinical effectiveness” — as if doctors needed the equivalent of a very-large-type book to be able to read what most of us can read with normal-sized type.
According to the article, “by 2000, more people were asking about keto, but most pediatric neurologists still would not prescribe it” — as if the parents needed the approval of their doctor to try it. You don’t need a prescription to buy food.
Thanks to Tim Beneke, Michael Bowerman, Alex Chernavsky, David Cramer, and Peter Couvares.
You are currently browsing the Seth's Blog weblog archives for November, 2010.
