One of my all time favourite bloggers, Oxford Neuroscientist Prof. Dorothy Bishop, or DeevyBee as she is known on Twitter has performed an amazing open access lecture focusing largely on theBad Neuroscience1 The Science of Bad Neuroscience misunderstanding of neuroscience (click down to the “Emanuel Miller Lecture” to play the video). The talk is incredibly informative and digestible, even those with no understanding of neuroscience or psychology whatsoever will take a great deal away. The problem of the poor understanding of neuroscience is one of the main reasons why I started this blog, so if you like this blog then you’ll love this lecture.

Bishop Lecture The Science of Bad Neuroscience

The talk begins with a reasoned explanation of how and when we should be sceptical of neuroscience research, Bishop goes on to cite 4 key reasons why certain kinds of scientific research will inevitbaly result in false-positives:

“The four horseman of the apocalypse”

1. Maturation – People develop naturally over time.

“There seems to be an implicit assumption that the brain, because it is a physical organ is somehow not going to change unless you give it some intervention – that it is there as a static thing. This is completely untrue… as evidenced by this series of images.”

Changes in the brain over time The Science of Bad Neuroscience

The brain changes naturally over time

2.  Practice effects – when people keep doing the same test again and again, they get better at it.

“…purely to do with the fact that you have got better at doing the test and nothing to do with your abilities… People forget that this can apply to language tests and thing like that. It also applies to some extent to the brain, often we don’t know how important this is because brain imaging is so new.. clearly if you get brain responses to novelty, that means if you do something twice – the first time round you will get different responses to the second time round when it is no longer novel”.

3. Regression to the mean  - a statistical artefact of longitudinal studies that is exacerbated if you select participants on the basis of a low score on a test (for example participants with developmental difficulties). Bishop does an outstanding job of explaining the problem at about 18 minutes in to the talk.

“Regression to the mean is as inevitable as death and taxes”

Campbell and Kenny (1999) A primer on regression artefacts

4. The placebo effect. This is the obvious consideration that continues to impact poorly designed research but according to Bishop, the three issues listed above could actually be having an even greater impact than the placebo effect.

The Solution?

Bishop explains that a control group is vital in order to achieve valid findings, but a control group alone is not enough, we should also be asking questions such as:

  • Are the groups randomly assigned – or is there some other factor at play?
  • Is the control group given an alternative treatment? If not, why not?
  • What causes drop out? People don’t tend to drop out at random and this can have a very big effect on results.
If something smells fishy, it probably is fishy.
Sometimes things just go wrong and currently in the field of brain imaging, an awful lot of things have been going wrong. This is well illustrated by the now famous study of the dead fish in the brain scanner. A result was found in two different trials where a dead fish was asked to determine facial expressions. For this reason, all research – but particularly abstract research such as brain imaging research – should be taken with a pinch of salt until the results have been replicated, ideally a few times.
Smells fishy dead fish brain scanner The Science of Bad NeuroscienceIf you like this lecture subscribe to Dorothy Bishop’s blog, it is one of those blogs that is so useful that frankly, it should be required reading for all concerned. Also, keep an eye out for Dorothy’s forthcoming paper where she will be publishing some of the ideas she presented in the Emanuel Miller Lecture.

References:

Weisberg, Deena Skolnick. (2008). The Seductive Allure of Neuroscience Explanations. Journal of Cognitive Neuroscience, 18 (3), 229-477 DOI: 10.1162/jocn.2008.20040

Campbell and Kenny (1999) A primer on regression artefacts

Return to Neurobonkers.com The Science of Bad Neuroscience
Tagged with:
 
  • fishslapper

    Here’s a question: At what point do you think neuroscience
    bloggers might realise the flaw inherent in citing an
    unreplicated fMRI investigation of a single deceased salmon as evidence challenging the
    replicability of the neuroimaging literature?

    • http://neurobonkers.com Neurobonkers

      The salmon example isn’t so much evidence as it is a humorous illustration, see
      http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2686233/?tool=pubmed for a review of the evidence.

      • http://www.facebook.com/apsaygin Ayşe Pinar Saygin

        That paper is about a different problem than the salmon and actually shows fMRI is not as problematic as sensationalist blogs try to make it out to be.

    • http://www.facebook.com/apsaygin Ayşe Pinar Saygin

      I agree with fishslapper. This just is patently absurd. Of course you can get spurious voxels if you scan one fish or one subject. Any published study will inherently have the replication you speak of by virtue of having multiple sessions and multiple subjects. Does anyone really think neuroimagers don’t understand what the fish thing means? It’s not funny because for people who don’t know what actually happens it has hurt our field’s credibility. To the point I have been told fMRI is pseudoscience. I don’t see how this is good for any area of science. I wish you would not perpetuate the damn fish. It’s completely irrelevant to how any neuroimaging research is done today.

  • Zac Perkins

    This article does a good job of showing how neuroscience can be misleading. Maturartion, the practic effect, regression to the mean, and the placebo effect are all things that effect the results of neuroscience and give false ideas. Control groups need to be used in neuroscience to help get more accurate results.

  • operp

    the link to bishop’s lecture doesn’t seem to work. anyone knows where to view it?

    • http://neurobonkers.com Neurobonkers

      I’ve passed the message on, technical problem apparently – hopefully it should be working again soon.

      • http://neurobonkers.com Neurobonkers

        The link is fixed now.

  • Andrei Foldes

    Id like to watch the lecture but its off limits, do you know where I could watch it?
    cheers, andrei

    • http://neurobonkers.com Neurobonkers

      The link is fixed now.

Looking for something?

Use the form below to search the site:


Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...