Archive for the 'evidence snobs' Category
A year and a half ago, the father of a friend of mine started taking Vitamin D3, 5000 IU/day at around 7 am — soon after getting up. That his regimen is exactly what I’d recommend (good dose, good time of day) is a coincidence — he doesn’t read this blog. He used to get 3 or 4 terrible colds every year, year after year. Since he started the Vitamin D3, he hasn’t gotten any. “A huge lifestyle improvement,” said my friend. His dad studied engineering at Caltech and is a considerable skeptic about new this and that.
Much more recently his mother changed the time of day she took her usual dose of Vitamin D3. For years she had been taking half in the morning (with a calcium supplement) and half at night. Two weeks ago she started taking the whole dose in the morning. Immediately — the first night — her sleep improved. She used to wake up every 2 hours. Since taking the Vitamin D3 in the morning, she has been waking up only every 3-6 hours. A few days ago, my friend reports she had “her best sleep in years”.
Sleep and immune function are linked in many ways beyond the fact that we sleep more when we’re sick. A molecule that promotes sleep turned out to be very close to a molecule that produces fever, for example. I found that when I did two things to improve my sleep (more standing, more morning light) I stopped getting colds. So it makes sense that a treatment that improves one (sleep or immune function) would also improve the other (immune function or sleep).
A few days ago I posted a link about a recent Vitamin D study that found no effect of Vitamin D on colds. The study completely neglected importance of time of day by giving one large injection of Vitamin D (100,000 IU) per month at unspecified time. I commented: “One more Vitamin D experiment that failed to have subjects take the Vitamin D early in the morning — the time it appears most likely to have a good effect.” These two stories, which I learned about after that post, support my comment. What’s interesting is that the researchers who do Vitamin D studies keep failing to take time of day into account and keep failing to find an effect and keep failing to figure out why. I have gathered 23 anecdotes that suggest that their studies are failing because they are failing to make sure their subjects take their Vitamin D early in the morning. Yet these researchers, if they resemble most medical researchers, disparage anecdotes. (Disparagement of anecdotes reaches its apotheosis in “evidence-based medicine”.) The same anecdotes that, I believe, contain the information they need to do a successful Vitamin D clinical trial. Could there be a serious problem with how Vitamin D researchers are trained to do research? A better approach would be to study anecdotes to get ideas about causation and then test those ideas. This isn’t complicated or hard to understand, but I haven’t heard of it being taught. If you understand this method, you treasure anecdotes rather than dismiss them (“anecdotal evidence”).
- Salem Comes to the National Institutes of Health. Dr. Herbert Needleman is harassed by the lead industry, with the help of two psychology professors.
- Climate scientists “perpetuating rubbish”.
- A humorous article in the BMJ that describes evidence-based medicine (EBM) as a religion. “Despite repeated denials by the high priests of EBM that they have founded a new religion, our report provides irrefutable proof that EBM is, indeed, a full-blown religious movement.” The article points out one unquestionable benefit of EBM — that some believers “demand that [the drug] industry divulge all of its secret evidence, instead of publishing only the evidence that favours its products.” Of course, you need not believe in EBM to want that. One of the responses to the article makes two of the criticisms of EBM I make: 1. Where is the evidence that EBM helps? 2. EBM stifles innovation.
- What really happened to Dominique Strauss-Kahn? Great journalism by Edward Jay Epstein. This piece, like much of Epstein’s work, sheds a very harsh light on American mainstream media. They were made fools of by enemies of Strauss-Kahn. Epstein is a freelance journalist. He uncovered something enormously important that all major media outlets — NY Times, Washington Post, The New Yorker, ABC, NBC, CBS (which includes 60 Minutes), the AP, not to mention French news organizations, all with great resources — missed.
From this comment (thanks, Elizabeth Molin) I learned of a British book called Testing Treatments (pdf), whose second edition has just come out. Its goal is to make readers more sophisticated consumers of medical research. To help them distinguish “good” science from “bad” science. Ben Goldacre, the Bad Science columnist, fulsomely praises it (“I genuinely, truly, cannot recommend this awesome book highly enough for its clarity, depth, and humanity”). He wrote a foreword. The main text is by Imogen Evans (medical journalist), Hazel Thornton (writer), Iain Chalmers (medical researcher), and Paul Glaziou (medical researcher, editor of Journal of Evidence-Based Medicine).
As a statistician, I was trained to think of randomized experimentation as representing the gold standard of knowledge in the social sciences, and, despite having seen occasional arguments to the contrary, I still hold that view, expressed pithily by Box, Hunter, and Hunter (1978) that “To find out what happens when you change something, it is necessary to change it.”
Box, Hunter, and Hunter (1978) (a book called Statistics for Experimenters) is well-regarded by statisticians. Perhaps Box, Hunter, and Hunter, and Andrew, were/are unfamiliar with another quote (modified from Beveridge): “Everyone believes an experiment except the experimenter; no one believes a theory except the theorist.” (more…)
I found out quite by accident WALNUTS get rid of [canker sores] quite quickly. The first sign of an ulcer I chew walnuts and leave the paste in my mouth for a little while (30 seconds or so).
The first time was by accident, my ulcers disappeared so quickly I knew it had to be something I ate. And the only thing I had eaten differently the past day was walnuts.
Flaxseed oil and walnuts differ in lots of ways but both are high in omega-3. My gums got much better around the time I started taking flaxseed oil. I neither noticed nor expected this; my dentist pointed it out. Several others have told me the same thing. Tyler Cowen’s gums got dramatically better. One reader started and stopped and restarted flaxseed oil, making it blindingly clear that the gum improvement is caused by flaxseed oil. There is plenty of reason to think the human diet was once much higher in omega-3. All this together convinces me that omega-3 can both prevent and cure canker sores. Not only that, I’m also convinced that canker sores are a sign of omega-3 deficiency. You shouldn’t just get rid of them with walnuts; you should change your diet. Omega-3 has other benefits (better brain function, less inflammation, probably others).
Let’s say I’m right about this — canker sores really are prevented and cured by omega-3. Then there are several things to notice.
1. Web facilitation. It was made possible by the internet. My initial interest in flaxseed oil came from reading the Shangri-La Diet forums. I didn’t have to read a single book about the Aquatic Ape theory; I could learn enough online. Tyler Cowen’s experience was in his blog. Eric Vlemmix contacted me by email. No special website was involved.
2. Value of self-experimentation. My flaxseed oil self-experimentation played a big part, although it had nothing to do with mouth health. These experiments showed dramatic benefits — so large and fast that something in flaxseed oil, presumably omega-3, had to be a necessary nutrient. Because of these results, I blogged about omega-3 a lot, which is why Eric emailed me about his experience.
3. Unconventional evidence. All the evidence here, not just the self-experimentation, is what advocates of evidence-based medicine and other evidence snobs criticize. Much of it is anecdotal. Yet the evidence snobs have, in this case, nothing to show for their snobbery. They missed this conclusion completely. Nor do you need a double-blind study to verify/test this conclusion. If you have canker sores, you simply drink flaxseed oil or eat walnuts and see if they go away. Maybe this omnipresent evidence snobbery is . . . completely wrong? Maybe this has something to do with the stagnation in health research?
4. Lack of credentials. No one involved with this conclusion is a nutrition professor or dentist or medical doctor, as far as I know. Apparently you don’t need proper credentials to figure out important things about health. Of course, we’ve been here before: Jane Jacobs, Elaine Morgan.
5. Failure of “trusted” health websites. Health websites you might think you could trust missed this completely. The Mayo Clinic website lists 15 possible causes — none of them involving omega-3. (Some of them, we can now see, are correlates of canker sores, also caused by lack of omega-3.) If canker sores can be cured with walnuts, the Mayo list of treatments reads like a list of scurvy cures from the Middle Ages. The Harvard Medical School health website is even worse. “Keep in mind that up to half of all adults have experienced canker sores at least once,” it says. This is supposed to reassure you. Surely something this common couldn’t be a serious problem.
6. Failure of the healthcare establishment. Even worse, the entire healthcare establishment, with its vast resources, hasn’t managed to figure this out. Canker sores are not considered a major health problem, no, but, if I’m right, that too is a mistake. They are certainly common. If they indicate an important nutritional deficiency (too little omega-3), they become very important and their high prevalence is a major health problem.
In an editorial about the effect of vitamin-mineral supplements in the prestigious American Journal of Clnicial Nutrition, the author, Donald McCormick, a professor of nutrition at Emory University, writes:
This study is a meta-analysis of randomized controlled trials that were previously reported. Of 2311 trials identified, only 16 met the inclusion criteria.
That’s throwing away a lot of data! Maybe, just maybe, something could be learned from other 2295 randomized controlled trials?
If we can believe a movie based on a true story, the doctors consulted by the family with an epileptic son in …First Do No Harm knew about the ketogenic diet but (a) didn’t tell the parents about it, (b) didn’t take it seriously, and (c) thought that irreversible brain surgery should be done before trying the diet, which was of course much safer. Moreover, these doctors had an authoritative book to back up these remarkably harmful and unfortunate attitudes. The doctors in …First, as far as I can tell, reflected (and still reflect) mainstream medical practice.
Certainly the doctors were evidence snobs — treating evidence not from a double-blind study as worthless. Why were they evidence snobs? I suppose the universal tendency toward snobbery (we love feeling superior) is one reason but that may be only part of the explanation.Â In the 1990s, Phillip Price, a researcher at Lawrence Berkeley Labs, and one of his colleagues were awarded a grant from the Environmental Protection Agency (EPA) to study home radon levels nationwide. They planned to look at the distribution of radon levels and make recommendations for better guidelines. After their proposal was approved, some higher-ups at EPA took a look at it and realized that the proposed research would almost surely imply that the current EPA radon guidelines could be improved. To prevent such criticism, the grant was canceled. Price wasÂ told by an EPA administrator that this was the reason for the cancellation.
This has nothing to do with evidence snobbery. But I’m afraid it may have a lot to do with how the doctors in …First Do No Harm viewed the ketogenic diet. If the ketogenic diet worked, it called into question their past, present, and future practices — namely, (a) prescribing powerful drugs with terrible side effects and (b) performing damaging and irreversible brain surgery of uncertain benefit. If something as benign as the ketogenic diet worked some of the time, you’d want to try it before doing anything else. This hadn’t happened: The diet hadn’t been tried first, it had been ignored. Rather than allow evidence of the diet’s value to be gathered, which would open them up to considerable criticism, the doctors did their best to keep the parents from trying it. Much like canceling the radon grant.
DOCTOR The diet is not an approved treatment.
MOTHER But there have been a lot of studies.
DOCTOR Those studies are anecdotal, not the kind of studies we base sound medical judgment on. Not double-blind studies.
DOCTOR I assume you know all the evidence in favor of the ketogenic diet is anecdotal. There’s absolutely no scientific evidence this diet works.
The doctor prefers brain surgery. When the diet is tried, it works beautifully (as it often does in real life). “What could have gone so horribly wrong with this whole medical system?” the mother writes the father.
At a reunion of Reed College graduates who majored in psychology, I gave a talk about self-experimentation. One question was what I thought of Evidence-Based Medicine. I said the idea you could improve on anecdotes had merit, but that proponents of Evidence-Based Medicine have been evidence snobs (which derives from Alex Tabarrok’s credit snobs). I meant they’ve dismissed useful evidence because it didn’t reach some level of purity. Because health is important, I said, ignoring useful information, such as when coming up with nutritional recommendations, is really unfortunate.
Afterwards, four people mentioned “evidence snobs” to me. (Making it the most-mentioned thing I said.) They all liked it. Thanks, Alex.
The Food and Drug Administration has settled a lawsuit related toÂ mercury in dental fillings. As part of the settlement, it will acknowledge that these fillings may harmÂ some people. This is from an email by someone behind the lawsuit:
To change FDA policy, we tried petitions, Congressional hearings, state fact sheet laws, Scientific Advisory Committee hearings, and letters galore — to no avail. So in the great American tradition, we sued. The case came to a head this spring. On April 22, working with Johann Wehrle and Gwen Smith, I filed a motion for an injunction before Judge Ellen Huvelle. Three sets of briefs later, the government and I presented our oral arguments on May 16. In a crucial ruling, Judge Huvelle ruled that our 11 plaintiffs — the diverse group
listed below — have standing. She said FDA should classify, and invited the two sides to mediate. On May 30, before Magistrate Judge John Facciola, Bob Reeves (who flew in from Lexington KY) and I hammered out an agreement with FDAÂ officials and lawyers.
The impact of the re-writing of its position on amalgam can hardly be understated. [A curious mistake: the writer means overstated.] FDAs website will no longer be cited by the American Dental Association in public hearings. FDA shows awareness of the key issues involved. As it prepares to classify amalgam, FDA has moved to a position of neutrality. Indeed, having repeatedly raised the question of amalgams risk to children, young women, and the immuno-sensitive persons in its website, I find it inconceivable that FDA will not in some way protect them in its upcoming rule.
Mercury fillings were once very common and are still common. UnfortunateÂ that it took a lawsuit to get the FDA to change.Â Judges have little or no relevant experience understanding scientificÂ papers.Â Scientific advisory panels have much more relevant experience. However, they suffer from a “purity” biasÂ – they are evidence snobs.
In a recent post I said that scientists are often much too dismissive. They are “evidence snobs,” Alex Tabarrok might say. A letter in the current issue of the American Journal of Clinical Nutrition criticizes a important example of just such dismissiveness:
In conclusion, whereas we agree that policy decisions should be evidence-based and not hasty, we do not agree that the evidence base [used to make those decisions] should be constrained to one type of study [long-term randomized controlled trials]â€”in particular, not to a study design that is inherently limited. Do we really want to wait perhaps decades for results of long-term RCTs, which almost certainly will not provide definitive evidence, while ignoring other relevant evidence involving shorter-term endpoints? An example is provided in the panel’s own summary statement (2). In lauding RCTs as the “gold standard for evidence-based decision making,” the panel proudly points to the fact that, even though folate was well known to decrease the risk of neural tube defects in animal studies, policy recommendations for folate supplementation to prevent neural tube defects were delayed while authorities waited some years for confirmation from RCTs. One can only wonder how many infants were born with neural tube defects while authorities waited.
“Proudly,” huh? Inclusion of that word shows how pissed the authors of the letter are — and rightly so. One author is Bruce Ames, a neighbor of mine, for whom I have great respect; another is Walter Willett, the Harvard epidemiologist. In 1998, Willett wrote a smart article challenging the popular belief that a low-fat diet is a good way to lose weight.
Here is part of the reply from the authors of the report that Ames et al. criticized:
It is important to note that our panel was not charged with asking whether vitamins and minerals play a role in human disease –a topic that occupies much of the letter by Ames et al, and for which observational evidence is indeed central — but, as a State-of-the Science Panel, was charged to reflect on the state of the available evidence for a treatment recommendation on the use of vitamins and minerals in the general population. For treatment decisions, the RCT is the established standard. No better proof of this principle can be found than in the RCTs reviewed in our report, which showed serious harm from vitamin ingestion in certain circumstances.
A less-than-reassuring answer. A commentator on my earlier post thought I should address the strongest arguments on the other side. I had trouble thinking of any. It’s hard to argue that less evidence is better. You can see that those who wrote this paragraph — some of the most prominent nutrition scientists in the country — were equally baffled.
I will revise my “common mistakes” article to mention the Ames et al. letter.