Seth's Blog

• meaning-based computing
• academic plagiarism. “One of my own students turned in a paper on “Great Expectations” which was an exact copy of Dorothy Van Ghent’s essay – an essay so celebrated that I recognized it right off and, at the first opportunity, raised the issue with my student. “Shit!” she said. “I paid seventy-five dollars for that.” “
• The dark side of fermentation. I am very pleased to see that Edward Jay Epstein is writing a book about the 9-11 Commission.

To my pleasant surprise, Mark Frauenfelder posted this call for volunteers. Will eating half a stick of butter per day or a similar amount of coconut fat improve your performance on arithmetic problems? Eri Gentry is organizing a simple trial to find out. The trial is inspired by my recent Quantified Self talk. Study details.

During the question period of my talk, I responded to a question about a trial with 100 volunteers by saying I would suggest starting with 2 volunteers. A reader has written to ask why.

What’s your reasoning behind suggesting only 2 volunteers to test the eating more butter results? You seem highly convinced earlier in the video, but if you were so convinced why not have a larger trial?

Because the trial will be harder than the people running it expect. If you’re going to make mistakes, make small ones.

This is my first rule of science: Do less. A grad student in English once told me that a little Derrida goes a long way and a lot of Derrida goes a little way. Same with data collection. A little goes a long way and a lot goes a little way. A tiny amount of data collection will teach you more than you expect. A large amount will teach you less.

My entire history of self-experimentation started with a small amount of data collection: An experiment about the effectiveness of an acne medicine. It was far more informative than I expected. My doctor was wrong, I was wrong — and it had been so easy to find out.

This may sound like I am criticizing Eri’s study. I’m not. What’s important is to do something, however flawed, that can tell you something you didn’t know. Maybe that should be the first rule, or the zeroth rule. It has the pleasant and unusual property of being easier than you might think.

Thanks to Carl Willat.

I asked an American friend who’s been in China for a year how the year had changed her. She told a story:

I was in a restaurant in Inner Mongolia. This guy was going around smashing things, throwing glasses. He was drunk. I was shocked. I expected a strong reaction: Get out of my restaurant! That’s not what happened. There was no strong reaction. The guy finally left and the staff cleaned up the mess he made. I’ve learned not to react strongly to unusual behavior.

I love this story. That travel changes your assumptions is hardly a new idea but this says it vividly and briefly.

From an excellent Atlantic article about John Ioannidis, who has published several papers saying that medical research is far less reliable than you might think:

A different oak tree at the site provides visitors with a chance to try their own hands at extracting a prophecy. “I [bring] all the researchers who visit me here, and almost every single one of them asks the tree the same question,” Ioannidis tells me . . . “‘Will my research grant be approved?’”

A good point. I’d say his main contribution, based on this article, is pointing out the low rate of repeatability of major medical findings. Until someone actually calculated that rate, it was hard to know what it was, unless you had inside experience. The rate turned out to be lower than a naive person might think. It was not lower than an insider might think, which explains lack of disagreement:

David Gorski . . . noted in his prominent medical blog that when he presented Ioannidis’s paper on [lack of repeatability of] highly cited research at a professional meeting, “not a single one of my surgical colleagues was the least bit surprised or disturbed by its findings.”

I also like the way Ioannidis has emphasized the funding pressure that researchers face, as in that story about the oak tree.  Obviously it translates into pressure to get positive results, which translates into overstatement.

I also think his critique of medical research has room for improvement:

1. Black/white thinking. He talks in terms of right and wrong. (“We could solve much of the wrongness problem, Ioannidis says, if the world simply stopped expecting scientists to be right. That’s because being wrong in science is fine.”) This is misleading. There is signal in all that medical research he criticizes; it’s just not as strong a signal as the researchers claimed. In other words the research he says is “wrong” has value. He’s doing the same thing as all those meta-analyses that ignore all research that isn’t of “high quality”.

2. Nihilism (which is a type of black/white thinking). For example,

How should we choose among these dueling, high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all.

I’ve paid a lot of attention to health-related research and benefited greatly.  Many of the treatments I’ve studied through self-experimentation were based on health-related research. An example is omega-3. There is plenty of research suggesting its value and this encouraged me to try it. Likewise, there is plenty of evidence supporting the value of fermented foods. That evidence and many other studies (e.g., of hormesis) paint a large consistent picture.

3. Bias isn’t the only problem, but, in this article, he talks as if it is. Bias is a relatively minor problem: you can allow for it. Other problems you can’t allow for. One is the Veblenian tendency to show off. Thus big labs are better than small ones, regardless of which would make more progress. Big studies better than small, expensive equipment better than cheap, etc. And, above all, useless is better than useful. The other is a fundamental misunderstanding about what causes disease and how to fix it. A large fraction of health research money goes to researchers who think that studying this or that biochemical pathway or genetic mechanism will make a difference — for a disease that has an environmental cause. They are simply looking in the wrong place. I think the reason is at least partly Veblenian: To study genes is more “scientific” (= high-tech = expensive) than studying environments.

Thanks to Gary Wolf.

I asked several Tsinghua students what they thought about Liu Xiaobo, the imprisoned Chinese dissident, winning the Nobel Peace Prize. There was a wide range of answers:

1. “It’s a sensitive subject,” said one student. And said no more.

2. “The Nobel Prize always seems to involve China,” said another student. Maybe she meant the Peace Prize in 1989 to the Dalai Lama and the more recent Literature prize to Gao Xingjian, but I’m not sure. Politely changing the subject.

3. “I don’t know much about what he stands for,” said another student (a freshman).

4. “Now is not the right time for his ideas. They would interfere with economic progress,” said a student who is a member of the Communist Party.

5. “Many people say because the European economy is bad, they gave the prize to someone who will never collect the money [because he's imprisoned],” said another student. She added that receiving the prize will be bad for Liu. Because it was “a great shame for China” (meaning the government), they will increase his prison sentence.