Archive for October, 2010

How to Lie with Meta-Analysis

Monday, October 11th, 2010

Michael Constans drew my attention to a Consumer Reports article about spinal surgery. According to the article, a popular type of spinal surgery called vertroplasty (involving cementing vertebrae together) doesn’t work and should be stopped:

Despite the popularity of the procedure, the American Academy of Orthopedic Surgeons has just released a guideline [actually a meta-analysis of available data] saying it doesn’t work, and shouldn’t be used.

They reviewed all the literature about the procedure, and found two good-quality studies (randomized controlled trials) that show vertebroplasty works no better than a fake (placebo) procedure. There were no clinically significant differences in pain or disability, they say.

No significant difference means it “doesn’t work”? At best, it’s absence of evidence. And it’s not even that because we don’t know what the rest of the research suggests. The meta-analysis might have ignored thirty studies; it doesn’t say how many studies were ignored. Nor does it say what “placebo” means. Patients are interested in pain relief. Whether the pain relief is “all in their head” or whatever hardly matters. From a patient’s view, and a clinician’s view, a better comparison is a group that gets another plausible treatment or no treatment. The meta-analysis reports one study that compared vertrebroplasty to “conservative” treatment, which makes more sense.

So how come vertebroplasty has been used so often? Other experts have recommended the treatment in the past, including the United Kingdom’s National Institute for Health and Clinical Excellence, which said most people got some pain relief from the procedure.

This suggests that ignoring of evidence and poor choice of comparison group made a difference.

In this case, it probably comes down to what you mean by significant pain relief. The US surgeons set a strict definition—they said a difference in pain relief of less than 2 points on a 10-point scale was not meaningful for patients. Smaller differences in pain relief were recorded in the studies, but, say the surgeons, they were not big enough to make a real difference.

I think patients would prefer to decide for themselves what degree of pain relief is meaningful.

The surgeons say the evidence against vertebroplasty is strong, and they don’t expect future studies to overturn their recommendations.

Haha! Two studies with inferior comparison groups that failed to find a difference — and ignoring an unknown number of studies and arbitrarily raising the bar — adds up “strong evidence against”!

Still the Guideline and Evidence Report is useful. It reviews a range of treatments, provides citations (so you can look much further), and isn’t wedded to the placebo-comparison group. It includes studies with other comparison groups; the article just doesn’t mention them.

And I completely agree with the article’s conclusion:

If you’re considering any type of surgery, ask your surgeon to show you data about how likely it is to solve your problem.

And don’t take your surgeon’s word for it that such data exists.

Where Does Oil Come From?

Sunday, October 10th, 2010

This fascinating article describes two ideas about oil production that were new to me: 1. It is made by microbes a long way down inside the Earth. 2. It is made by nuclear reactions going on in the middle of the earth.

The first idea came from a Cornell geologist named Thomas Gold. According to the article, “some geologists were so incensed by Gold’s ideas they petitioned to have the government remove all mention of it from the nation’s libraries.” That is so strange (and no source cited) you might think the whole article is made-up but Gold explained his ideas here (short) and here (long).

Thanks to Carl Willat.

Strong Light and Cancer

Sunday, October 10th, 2010

From an excellent article about light pollution (not online) by David Owen in the 20 Aug 2007 issue of The New Yorker:

Richard Stevens, a cancer epidemiologists at the University of Connecticut Health Center, in Farmington, has suggested a link between cancer and the “circadian disruption” of hormones caused by artificial lighting. Early in his career, Stevens was one of many researchers struck by the markedly high incidence of breast cancer among women in the industrial world, in comparison with those in developing countries, and he at first supported the most common early hypothesis, which was that the cause must be dietary. Yet repeated studies found no clear link to food. In the early eighties, Stevens told me recently, “I literally woke up in the middle of the night — there was a street lamp outside the window, and it was so bright that I could almost read in my bedroom — and I thought, Could it be that? A few years later, he persuaded the [directors] of the Nurses’ Health Study . . . to add questions about nighttime employment, and the study subsequently revealed a strong association between working the night shift and an increased risk of breast cancer. [The researchers] wrote, “We hypothesize that the potential primary culprit for this observed association is the lack of melatonin, a cancer-protective agent whose production is severely diminished in people exposed to light at night.”

Exposure to strong light at night reduces the amplitude of your circadian rhythms. That causes a thousand changes. To decide that one of them (“lack of melatonin”) is the one that matters is highly premature. If reducing circadian amplitude increases cancer, it follows that getting more light during the day — which surely increases circadian amplitude — will reduce cancer.

The article also says:

Growing numbers of us pass most of our waking hours “in a box, looking at a box,” as Dave Crawford put it . . . Fewer and fewer of us spend much time outside at all, except in automobiles.

I have measured the light inside cars (front seats) several times and found it is quite strong (you are close to a big window). If the article is arguing that night light is bad and causes cancer, I am unconvinced. Night light exposure and daylight exposure are confounded — people who work night shifts get more night light and less daylight.
The Nurses Study paper: E Schernhammer, K Schulmeister. Light at night and cancer risk. Photochemistry and Photobiology, 2004, Vol 79, Iss 4, pp 316-318.

Chinese Text-Message Censorship

Saturday, October 9th, 2010

An American friend wondered about Chinese reaction to Liu Xiaobo winning the Nobel Peace Prize. So I sent a text message to four Chinese friends: “Did u know chinese dissident won nobel peace prize?”

Six hours later I hadn’t received any replies. I phoned one of them. She hadn’t received my message. I saw it hadn’t gone through: “Unable to send message”. I tried again. Failed again. Then I tried to send “did u know chinese dissident won prize?” Success. I tried the earlier message, with “nobel peace prize”. Fail. Tried the shorter message. Success.

Messages almost never fail, so it was clearly censorship. One of my friends said the same about messages in Chinese: “We can’t send email or text messages” about it. She had heard about the prize from her classmates. She didn’t know how they had learned about it. Another Chinese friend read it on a website. Later it was gone.

I thought of the boy who cried wolf. One day there was a wolf. But no one believed him.

Dairy Consumption and Health

Friday, October 8th, 2010

Two studies of the effect of dairy consumption on health have recently appeared. Both suggest it is healthy. One of them– a prospective study where about 1500 people were followed for 16 years — found no association of dairy intake with overall mortality but did find a protective effect of full-fat dairy against heart disease. The study considered lots of possibilities and the authors write ” it is important to take into account the large number of comparisons considered in this study and thus we cannot rule out the possibility that the protective association between full-fat dairy intake and cardiovascular mortality was due to chance.”

I mentioned this study earlier. It gains more credence because of the other study, which is a meta-analysis. The second study found protective effects of dairy products on several outcomes, including overall mortality:

Meta-analyses suggest a reduction in risk in the subjects with the highest dairy consumption relative to those with the lowest intake: 0.87 (0.77, 0.98) for all-cause deaths, 0.92 (0.80, 0.99) for ischaemic heart disease, 0.79 (0.68, 0.91) for stroke and 0.85 (0.75, 0.96) for incident diabetes.

This is good news for me since I eat yogurt and butter every day.

Thanks to Peter Spero.