The Science of Bad Neuroscience

One of my all time favourite bloggers, Oxford Neuroscientist Prof. Dorothy Bishop, or DeevyBee as she is known on Twitter has performed an amazing open access lecture focusing largely on the misunderstanding of neuroscience (click down to the “Emanuel Miller Lecture” to play the video). The talk is incredibly informative and digestible, even those with no understanding of neuroscience or psychology whatsoever will take a great deal away. The problem of the poor understanding of neuroscience is one of the main reasons why I started this blog, so if you like this blog then you’ll love this lecture.

Click here to download the slides.

The talk begins with a reasoned explanation of how and when we should be sceptical of neuroscience research, Bishop goes on to cite 4 key reasons why certain kinds of scientific research will inevitbaly result in false-positives:

“The four horseman of the apocalypse”

1. Maturation – People develop naturally over time.

“There seems to be an implicit assumption that the brain, because it is a physical organ is somehow not going to change unless you give it some intervention – that it is there as a static thing. This is completely untrue… as evidenced by this series of images.”

The brain changes naturally over time

2.  Practice effects – when people keep doing the same test again and again, they get better at it.

“…purely to do with the fact that you have got better at doing the test and nothing to do with your abilities… People forget that this can apply to language tests and thing like that. It also applies to some extent to the brain, often we don’t know how important this is because brain imaging is so new.. clearly if you get brain responses to novelty, that means if you do something twice – the first time round you will get different responses to the second time round when it is no longer novel”.

3. Regression to the mean  – a statistical artefact of longitudinal studies that is exacerbated if you select participants on the basis of a low score on a test (for example participants with developmental difficulties). Bishop does an outstanding job of explaining the problem at about 18 minutes in to the talk.

“Regression to the mean is as inevitable as death and taxes”

Campbell and Kenny (1999) A primer on regression artefacts

4. The placebo effect. This is the obvious consideration that continues to impact poorly designed research but according to Bishop, the three issues listed above could actually be having an even greater impact than the placebo effect.

The Solution?

Bishop explains that a control group is vital in order to achieve valid findings, but a control group alone is not enough, we should also be asking questions such as:

• Are the groups randomly assigned – or is there some other factor at play?
• Is the control group given an alternative treatment? If not, why not?
• What causes drop out? People don’t tend to drop out at random and this can have a very big effect on results.

If you like this lecture subscribe to Dorothy Bishop’s blog, it is one of those blogs that is so useful that frankly, it should be required reading for all concerned. Also, keep an eye out for Dorothy’s forthcoming paper where she will be publishing some of the ideas she presented in the Emanuel Miller Lecture.

References:

Weisberg, Deena Skolnick. (2008). The Seductive Allure of Neuroscience Explanations. Journal of Cognitive Neuroscience, 18 (3), 229-477 DOI: 10.1162/jocn.2008.20040

Campbell and Kenny (1999) A primer on regression artefacts