Voodoo Correlations in Social Neuroscience
Sunday, December 28th, 2008Few scientific papers arouse emotion in reviewers and editors but this one — by my friend and collaborator Hal Pashler and his colleagues — must have because they allowed the use of voodoo in the title instead of spurious. Here is part of the abstract:
The newly emerging field of Social Neuroscience has drawn much attention in recent years, with high-profile studies frequently reporting extremely high (e.g., >.8) correlations between behavioral and self-report measures of personality or emotion and measures of brain activation obtained using fMRI. We show that these correlations often exceed what is statistically possible . . . Social-neuroscience method sections rarely contain sufficient detail to ascertain how these correlations were obtained. We surveyed authors of 54 articles that reported findings of this kind to determine the details of their analyses. More than half acknowledged using a strategy that computes separate correlations for individual voxels, and reports means of just the subset of voxels exceeding chosen thresholds. We show how this non-independent analysis grossly inflates correlations, while yielding reassuring-looking scattergrams. This analysis technique was used to obtain the vast majority of the implausibly high correlations in our survey sample.
The papers shown to be misleading appeared in such journals as Science and Nature.







