Seth's Blog

An area of science is an ecosystem in the sense that research builds on other research. In an ordinary ecosystem the animals and plants need each other. Different organisms add different things. Their contributions fit together. In a healthy scientific ecosystem, different types of research add different things and fit together. (more…)

One hundred years ago (January, 1912), at the annual meeting of the Geological Association in Frankfurt, Germany, Alfred Wegener, a meteorologist, presented his theory of continental drift for the first time. It was almost uniformly dismissed by geologists. One of them called it “mere geopoetry”. Much later, he was proved right.

To me, this is a classic example of the power of what I call insider/outsiders. Wegener had a great deal of scientific training, including a Ph.D. in astronomy. Unlike professional geologists, however, (a) he had the freedom to say whatever he wanted about geology without endangering his job (as a meteorologist) or prospects for advancement and (b) was under no pressure to publish. He could spend as much time on his theory as he wanted.

Peter A. Lawrence is a British biologist who has written several papers about problems with the way biology and other areas of science are now done. In this interview a year ago he summarizes his complaints:

• Scientific publication “has become a system of collecting counters for particular purposes – to get grants, to get tenure, etc. – rather than to communicate and illuminate findings to other people. The literature is, by and large, unreadable.” There is far too much counting of papers.
• “There’s a reward system for building up a large group, if you can, and it doesn’t really matter how many of your group fail, as long as one or two succeed. You can build your career on their success.” If you do something on your own it is viewed with suspicion.
• There is too much emphasis on counting citations. “If you work in a big crowded field, you’ll get many more citations. . . . This is independent of the quality of the work or whether you’ve contributed anything. [There is] enormous pressure on the journals to accept papers that will be cited a lot. And this is also having a corrupting effect. Journals will tend to take papers in medically-related disciplines, for example, that mention or relate to common genetic diseases. Journals from, say, the Cell group, will favor such papers when they’re submitted.”
• Grant writing takes too much time — e.g., 30-40% of your time. “There is an enormous increase in bureaucracy – form
  filling, targeting, assessment, evaluations. This has gone right through society, like the Black Death!”
• “Science is not like some kind of an army, with a large number of people who make the main steps forward together. You need to have individually creative people who are making breakthroughs – who make things different. But how do you find those people? I don’t think you want to have a situation in which only those who are competitive and tough can
  get to the top, and those who are reflective and retiring would be cast aside.” I’ve said something similar: Science is like single ants wandering around looking for food, not like a trail of ants to and from a food source. The trail of ants is engineering.

I agree. I would add that I think modern biology is far too invested in the idea that genes cause disease and that studying genes will help reduce human suffering. I think the historical record (the last 30 years) shows that this is not a promising line of work — but modern biologists cannot switch course.

What explains the depressing facts Lawrence points out? I think it is something deep and impossible to change: Science and job don’t mix well. The demands of any job and the demands of science are not very compatible. Jobs are about repetition. Science is the opposite. Jobs demand regular output. Science is unpredictable. However, jobs and science  overlap in terms of training: Both benefit from specialized knowledge. They also overlap in terms of resources: More resources (e.g., better tools) will usually help you do your job better, likewise with science. So we have two groups (insiders — professional scientists — and outsiders — everyone else). Both groups have big advantages and big disadvantages relative to the other. In the last 50 years, the insiders have been “winning” in the sense of doing better work. Their advantages of training and resources far outweighed the problems caused by the need for repetition and predictability. But now — as I try to show on this blog — outsiders are catching up and going ahead because the necessary training and tools have become much more widely available (e.g., tools have become much cheaper). And, as Lawrence emphasizes, professional science has gotten worse.

Epidemiology has lots of critics. In this article, for example, it is called “lying on a grand scale.” Every critique I have read has ignored history. Epidemiologists have been right about two major issues: 1. Heavy smoking causes lung cancer. 2. Folate deficiency causes birth defects. In both cases, the first evidence was epidemiological. Another example is John Snow’s conclusion about the value of clean water. In my experience, epidemiologists often overstate the strength of their evidence (as do most of us) but overstatement is quite different from having nothing worth saying.

Let’s look at an example. Many people think osteoporosis is due to lack of calcium. Bones are made of calcium, right? The epidemiology of hip fractures is clear. In spite of the conventional idea, the rate of hip fracture has been highest in places where people eat a lot of calcium, such as Sweden, and lowest in places where they eat little, such as Hong Kong. (For example.) In other words, the epidemiology flatly contradicted the conventional idea. This was apparently ignored by nutrition experts (everyone knows correlation does not equal causation) who advised millions of people, especially women, to take calcium supplements  to avoid osteoporosis. Millions of people followed (and follow) that advice.

Thanks to a recent meta-analysis we now know that experiments and better data firmly support the earlier epidemiology, which suggested that calcium supplements are dangerous. Here are its main conclusions:

In meta-analyses of placebo controlled trials of calcium or calcium and vitamin D, complete trial-level data were available for 28,072 participants from eight trials of calcium supplements and the WHI CaD participants not taking personal calcium supplements. . . .Calcium or calcium and vitamin D increased the risk of myocardial infarction (relative risk 1.24 (1.07 to 1.45), P = 0.004) and the composite of myocardial infarction or stroke (1.15 (1.03 to 1.27), P = 0.009). . . . A reassessment of the role of calcium supplements in osteoporosis management is warranted.

If the epidemiology had been taken more seriously, many heart attacks might have been avoided.

Is this an “anecdote” — a single example — proving nothing? Here’s how you can check. Randomly select a meta-analysis of epidemiological studies. Thousands have been done. Then ask if the results summarized in the meta-analysis appear random. Better yet, randomly pick two meta-analyses. Suppose the first summarizes 5 studies and the second summarizes 6. If the 11 results were shuffled together, how well could you assign them correctly?

At the Quantified Self blog, in response to a video of me talking about QS and the Ancestral Health Symposium (paleo), someone named Colin made the following comment:

Very interesting talk. I am just curious how someone can claim a study conducted with a sample size of one is “100 times better” than someone else’s study. I do not know anything about the other study mentioned, but I do know that a study based on n=1 cannot be considered scientific proof. And sure, he hears from people who have lost weight drinking the sugar water he prescribed, but it is quite possible there are 100 times as many people who didn’t email him because they didn’t see any positive results and decided to try something else. I think the QS stuff is very interesting and helpful on a personal level, but it seems like a stretch to generalize your results to others.

I responded:

I have two responses.

1. Sample size isn’t everything. Sure, a study with n=1 isn’t “scientific proof”. Nor is any other study, in my experience. “Scientific proof” has always required many studies. New scientific ideas have very often started with n = 1 experiments or observations. Later, larger experiments or observations were done. Both — the initial n=1 observation and the later n = many observations — were necessary for the new idea to be discovered and confirmed.

2. The history of biology teaches there are few exceptions to general rules. See any biology textbook. For example, a textbook might say “lymphocytes fight germs”. This means no serious exceptions have ever been found to that rule. So, as matter of biological history, the person who managed to figure out what one particular lymphocyte does turned out to have figured out what they all do. Biology textbooks have thousands of statements like “lymphocytes fight infection” meaning that this sequence of events (you can generalize from one to all, or nearly all) has happened thousands of times. There is no shadow hidden history of biology that teaches otherwise.