A few weeks ago I blogged about a leukemia doctor’s disapproval of self-experimentation (“you won’t learn anything and others won’t learn from it, either”). What I wrote was reposted at The Health Care Blog, where it elicited this comment (by “rbar”):
Sigh. Mr Roberts did it again, he simply does not (want to) understand that anecdotal evidence is of little value (let me give you an example: I self experiment with traffic signals; I noted that I can considerable cut down on travel times when ignoring red lights and stop signs; there are no drawbacks whatsoever, no one get hurts, and even my gas mileage/carbon footprint got better) .
Individuals who have similar questions as Mr. Roberts should look up the following key words, because they may understand why controlled studies are far superior to anecdotal evidence:
-regression to the mean
-misattribution error [apparently rbar means error in determining the cause of a change]
-self limited conditions/natural fluctuation of chronic conditions
-and in terms of drawbacks of experimentation: primum non nocere, and also the fact that anecdotal evidence adds relatively little to humanity’s knowledge base
Does all that mean that patients should not be well informed, active and making suggestions to their treating physicians? Of course absolutely not. Being knowledgeable about one’s condition is different from self experimentation. Is that intellectually challenging?
One reply to this comment said we should be aggregating data across patients. “I believe Mr. Roberts is alluding to the power of aggregating real-world data across patients to generate insights into what may and may not work, not to giving undue weight to any single anecdotal case.” No, I was looking at it from the point of view of the self-experimenting patient. If you have a health problem, and you can measure it often (daily, weekly) you can find out what works faster than your doctor — often much faster. You can test many more possible solutions. This is what Richard Bernstein taught the whole world of diabetes, starting in the 1960s, when he pioneered home blood glucose testing. Apparently rbar also objects to that.
Rbar’s comment is dismissive (“Sigh”, “Is that intellectually challenging?”) and partly obscure (“ignoring stop signs and stoplights” — huh?). Because patients who self-experiment may make “misattribution errors” they shouldn’t self-experiment? That’s like saying because people may make reasoning errors they shouldn’t reason.
The true meaning of rbar’s comment may be hidden in his statement that it’s okay for patients to “make suggestions to their treating physician.” Which shows who he thinks should be boss in the doctor-patient relationship. When a patient self-experiments, the doctor is no longer boss. Maybe rbar is a doctor. Maybe he feels threatened by self-experimentation. If so, I hope he’s right.
More A later reply to rbar put it well: ” Your list of possible pitfalls . . . is similar to lists I remember seeing back in graduate school in various research handbooks. I do not see how you go from the fact that these effects and errors are possible to the conclusion that the whole endeavor isn’t worthwhile.”