Seth's Blog

A new article in The New Yorker by Jonah Lehrer is about declines in the size of experimental or quasi-experimental effects over time. For example, Jonathan Schooler, an experimental psychologist, found that if subjects are asked to describe a face soon after seeing it their later memory for the face is worse. As Schooler continued to study the effect, it appeared to get weaker. The article also describes examples from drug trials (a anti-psychotic drug appeared to become weaker over 15 years) and ecology (the effect of male symmetry on mating success got weaker over years).

It’s nice to see an ambitious unconventional article. I blogged a few weeks ago about difficulties replicating the too-many-choices effect. Difficulty of replication and the decline effect are the same thing. I could do what Jared Diamond does in Collapse: give a list of five or six reasons why this happens. (Judging by this paper, the effect, although real, is much weaker than you’d guess from Lehrer’s article.) For example, the initial report has much more flexibility of data analysis than later reports. Flexibility of analysis allows researchers to increase the size of effects.

A long list of reasons would miss a larger point (as Diamond does). A larger point is this: Science (search for truth) and profession (making a living) are not a good fit. In a dozen ways, the demands of a scientist’s job get in the way of finding and reporting truth. You need to publish, get a grant, please your colleagues, and so on. Nobody pays you for finding the truth. If that is a goal, it is several goals from the top of the list. Most jobs have customers. If a wheelwright made a bad wheel, it broke. Perhaps he had to replace it or got a bad reputation. There was fast powerful feedback. In science, feedback is long-delayed or absent. Only long after you have been promoted may it become clear anything was wrong with the papers behind your promotion. The main customers for science are other scientists. The pressure to have low standards — and thus appear better to promotion committees and non-scientists — is irresistible. Whereas if Wheelwright Y makes better wheels than Wheelwright X, customers may notice and Wheelwright Y may benefit.

There are things about making science a job that push scientists toward the truth as well, such as more money and time. When science is a job, a lot more research gets done. Fine. But how strong are the forces against finding truth? I was never surprised by the replication difficulties Lehrer writes about. I had heard plenty of examples, knew there were many reasons it happened. But I was stunned by the results of my self-experimentation. I kept finding stuff (e.g., breakfast disturbs sleep, butter improves brain function) that contradicted the official line (breakfast is the most important meal of the day, butter is dangerous). Obviously I had a better tool (self-experimentation) for finding things out. The shock was how many things that had supposedly been found out were wrong. Slowly I realized how much pressure career demands place on scientists. It is no coincidence that the person most responsible for debunking man-made global warming, Stephen McIntyre, is not a professional climatologist (or a professional scientist in any other area). Unlike them, he can say whatever he wants.

Thanks to Peter Couvares.

More In his blog, failing to see the forest for the trees, Lehrer says we must still believe in climate change (presumably man-made): climate change and evolution by natural selection “have been verified in thousands of different ways by thousands of different scientists working in many different fields.” Charles Darwin, like McIntyre, was an amateur, and therefore could say whatever he wanted.

Slate has recently published several articles on self-tracking. “How should we use data to improve our lives?” is a nice way to frame it. By data, the author, Michael Egger, means data we collect ourselves — leaving the traditional collectors of data, such as government and scientists, out of the loop (act –> collect data –> act). The first person to close the loop like that was Richard Bernstein, who measured his own blood sugar levels several times/day — omitting his doctor, who had measured Bernstein’s blood sugar level once/month, out of the feedback loop. The consequences were huge. Bernstein’s health got much better. And the treatment of diabetes changed forever when what Bernstein did became common. Hanna Rosin wrote about tracking her blood sugar levels in an article with a completely misleading subtitle (“Diabetes has forced me to become a self-tracker, and I can’t stand it”).

Another article — titled ”Living the Quantified Life: Some of the most inspiring self-tracking projects” but promoted as “The guy who eats a half-stick of butter a day and other strange ‘self-trackers’ ” — is about three examples of self-tracking: my butter research, the benefits of categorizing one’s possessions, and Jon Cousins’s discovery that telling other people his mood greatly improved it.

Slate is running a contest about this:

We are looking for great ways that we can collect and analyze data to improve our lives. You can submit your idea by clicking the button below. The deadline for submitting ideas has been extended until Wednesday, Dec. 8. We’ll be tracking your most interesting ideas throughout the month. And don’t forget to vote on the proposals you like best. We’ll take a closer look at the three top-vote-getting ideas and write about them.

I have liked the New Yorker coverage of Wikileaks but my favorite bit was this comment:

The world is divided in many “twos” and I would add another one. Those who are for and those who are against Wikileaks. I will try to describe each group. AGAINST: If a part or the full of your daily life deals with corruption, war crimes, extortion, blackmailing, malfeasance, bribery cover-up, then Assange is definitely a nightmare for you. You surely would like to get rid of him so that you can carry on with your evil. FOR: If you are an honest person, with high principles and impeccable conduct, a person who believes in true justice for each and every single one of the citizens, a person who supports education of the masses so that they can take informed decisions instead of being daily brainwashed and lied to by the Mainstream Media, then you are not afraid of the truth, you love the truth and you want to protect the innocent.

This is the modern version of The Emperor Has No Clothes in which it took a child to point out the obvious. No serious journalist could say this. As far as I can tell, no serious journalist has. It is too simple. Too disrespectful. Too sentimental. But it is surely true.

• animals that live with us are getting fat
• Drug trials in poor countries. “The F.D.A. does so little monitoring [in these countries] that the [drug] companies can pretty much do and say what they want.”
• New York obesity researcher re-invents numerical integration

Thanks to Gary Wolf and Bruce Charlton.

The head of the Baltimore school system, Andres Alonso, is fond of saying this:

Kids come as is and it’s our job to engage them.

I couldn’t agree more. In Totto-Chan, Tetsuko Kuroyanagi described meeting the headmaster of her new school. They had a conversation lasting hours. She remembered it as the most anyone had ever listened to her.
The full English title of Totto-Chan is Totto-Chan: The Little Girl at the Window. “At the window” is a Japanese term for failure — businessmen judged incompetent were seated near the window. At her previous school, Kuroyanagi had been a misfit and expelled — for, among other things, opening and closing her desk too often.