One of my all time favourite bloggers, Oxford Neuroscientist Prof. Dorothy Bishop, or DeevyBee as she is known on Twitter has performed an amazing open access lecture focusing largely on the misunderstanding of neuroscience (click down to the “Emanuel Miller Lecture” to play the video). The talk is incredibly informative and digestible, even those with no understanding of neuroscience or psychology whatsoever will take a great deal away. The problem of the poor understanding of neuroscience is one of the main reasons why I started this blog, so if you like this blog then you’ll love this lecture.
The talk begins with a reasoned explanation of how and when we should be sceptical of neuroscience research, Bishop goes on to cite 4 key reasons why certain kinds of scientific research will inevitbaly result in false-positives:
“The four horseman of the apocalypse”
1. Maturation – People develop naturally over time.
“There seems to be an implicit assumption that the brain, because it is a physical organ is somehow not going to change unless you give it some intervention – that it is there as a static thing. This is completely untrue… as evidenced by this series of images.”
2. Practice effects – when people keep doing the same test again and again, they get better at it.
“…purely to do with the fact that you have got better at doing the test and nothing to do with your abilities… People forget that this can apply to language tests and thing like that. It also applies to some extent to the brain, often we don’t know how important this is because brain imaging is so new.. clearly if you get brain responses to novelty, that means if you do something twice – the first time round you will get different responses to the second time round when it is no longer novel”.
3. Regression to the mean - a statistical artefact of longitudinal studies that is exacerbated if you select participants on the basis of a low score on a test (for example participants with developmental difficulties). Bishop does an outstanding job of explaining the problem at about 18 minutes in to the talk.
“Regression to the mean is as inevitable as death and taxes”
Campbell and Kenny (1999) A primer on regression artefacts
4. The placebo effect. This is the obvious consideration that continues to impact poorly designed research but according to Bishop, the three issues listed above could actually be having an even greater impact than the placebo effect.
Bishop explains that a control group is vital in order to achieve valid findings, but a control group alone is not enough, we should also be asking questions such as:
- Are the groups randomly assigned – or is there some other factor at play?
- Is the control group given an alternative treatment? If not, why not?
- What causes drop out? People don’t tend to drop out at random and this can have a very big effect on results.
Weisberg, Deena Skolnick. (2008). The Seductive Allure of Neuroscience Explanations. Journal of Cognitive Neuroscience, 18 (3), 229-477 DOI: 10.1162/jocn.2008.20040
Campbell and Kenny (1999) A primer on regression artefacts
A paper published yesterday in PLOS ONE by the Richard Wiseman et al (the king of weird and wonderful psychology experiments) has apparently disproved the long-standing theory that direction of eye gaze can indicate lying. The theory which forms part of the bed-rock of the controversial offshoot of psychology called ‘Neuro-linguistic programming’ (NLP) has in fact never actually been experimentally researched until now. The study found absolutely no correlation between eye gaze and lying. Considering that this theory has become such a staple of popular psychology, it really is astounding that this was not discovered sooner.
From the abstract:
“Proponents of Neuro-Linguistic Programming (NLP) claim that certain eye-movements are reliable indicators of lying. According to this notion, a person looking up to their right suggests a lie whereas looking up to their left is indicative of truth telling. Despite widespread belief in this claim, no previous research has examined its validity. In Study 1 the eye movements of participants who were lying or telling the truth were coded, but did not match the NLP patterning. In Study 2 one group of participants were told about the NLP eye-movement hypothesis whilst a second control group were not. Both groups then undertook a lie detection test. No significant differences emerged between the two groups. Study 3 involved coding the eye movements of both liars and truth tellers taking part in high profile press conferences. Once again, no significant differences were discovered. Taken together the results of the three studies fail to support the claims of NLP. The theoretical and practical implications of these findings are discussed.”
Wiseman, R., Watt C., Brinke, L., Porter, S., Couper, S., & Rankin, C. (2012). The Eyes Don’t Have It: Lie Detection and Neuro-Linguistic Programming PLOS ONE : 10.1371/journal.pone.0040259
A new documentary discusses a bizarre affliction that has been widely (and uncritically) reported in the media to affect around a dozen of the approximately one million Japanese tourists who visit Paris each year. Paris Syndrome is said to occur when a combination of factors leave tourists with a particularly severe case of culture shock.
Symptoms are purported to include:
‘acute delusional states, hallucinations, feelings of persecution (perceptions of being a victim of prejudice, aggression, or hostility from others), derealization, depersonalization, anxiety, and also psychosomatic manifestations such as dizziness, tachycardia (and) sweating’ - Wikipedia
Due to the relatively microscopic numbers reported, it seems to me to be all too likely that Paris has become the unfortunate victim of an illusory correlation. After all, twelve out of one million is a number that doesn’t immediately strike me as particularly statistically significant, to say the least. To put this figure in to perspective, seven per thousand of the adult population are expected to suffer a schizophrenic episode at some point in their life. That is the equivalent of seven thousand per million, a number which suddenly makes twelve per million sound much smaller indeed.
Unfortunately, there is little by the way of well documented reports or case studies of Paris Syndrome and Professor Hiroaki Ota, the author of the original report1 published in a French psychiatric magazine, appears somewhat impossible to contact. It is certainly stirring to entertain the thought that perhaps something as seemingly benign (to a modern western generation) as a holiday to Paris, could spark a breakdown so severe it requires the victim to seek refuge and be accompanied back to their home country. In many ways there is some logic to the idea, there has certainly been plenty of theorising by the media and the internet’s hivemind. In reality however, the explanation for this peculiar condition may come down to chance as much as it does to the cultural relationship between Parisians and their Japanese guests.
1. Viala, A., Ota, H., Vacheron, M.N., Martin, P., & Caroli, F. (2004). “Les Japonais en voyage pathologique à Paris : un modèle original de prise en charge transculturelle”. Neuvure de journal Psychiatrie, 5, 31-34
NB: I couldn’t figure out how to access the article referenced by the documentary, if anyone has access and the time and inclination to translate it from French I’d greatly appreciate it.
Exercise ‘no help for depression’, research suggests, reads todays BBC headline (despite the cited quote appearing not once in the research paper published today in the BMJ). The Guardian lead with: Exercise doesn’t help depression, study concludes. This comes as shockingly blunt advice after we have been told for years that exercise helps treat depression and a wide range of evidence has come to support this view.
So, what did the study actually find? Two groups of depressed individuals were kept on their standard treatment plan and one of those groups was mildly encouraged to do more exercise through a few short telephone calls and a couple of face to face meetings. There was no minimum amount of exercise required for inclusion in the study, nor were any facilities for exercise provided. Over half of the participants were on anti-depressant medication that may provide some of the benefits of exercise alone, thus negating the benefit of exercise on self reported happiness.
“The aspiration was for the participants to engage in moderate or vigorous activity for 150 minutes a week in bouts of at least 10 minutes, but if that seemed unrealistic then the facilitator encouraged any increase in physical activity, whatever the intensity. The intervention programme comprised an initial hour long face to face assessment session followed by two short telephone contacts, then a further face to face meeting for half an hour. Over the course of 6-8-months, the physical activity facilitator offered up to eight further telephone contacts and one more face to face half hour meeting.”
Chalder, M. Wiles, N. Campbell, J. Hollinghurst, S. Haase, A. Taylor, A. Fox, K. Costelloe, C. Searle, A. Baxter, H. Winder, R. Wright, C. Turner, K., & Calnan, M. Lawlor, D. Peters, T. Sharp, D. Montgomery, A. Lewis, G. (2012). Facilitated physical activity as a treatment for depressed adults: randomised controlled trial. BMJ, 344 DOI: 10.1136/bmj.e2758
At the four month follow up, exactly the same number of people in the treatment group had participated in physical activity as had done so in the control group (though it should be noted that there were seventeen more people in the control group than in the treatment group). Over the course of the study there was only a fifteen percent difference in the amount of exercise between the two groups! This study shows that the current exercise based treatment plan of telling people to exercise is not effective. It does not assess the outcomes of enabling people to exercise, or indeed of actually exercising.
Update: A follow up on this study in Scientific American comes to the same conclusion.
John Bargh, a Professor of Psychology and Cognitive Science at Yale University has written a blog post that’s currently receiving a thorough dressing down by the academic community. The title of the blog post, “Nothing in Their Heads” is a scathing ad-hom attack on a research group that failed to replicate his research. The opening gambit is an attack on, well the entire academic community.
“Scientific integrity in the era of pay-as-you-go publications and superficial online science journalism.”
It’s like a lesson in how to alientate your audience. “Superficial online science journalism” apparently refers to Ed Yong’s critique of his research. Yong isn’t exactly a light weight, as well as his Discover magazine blog he writes for New Scientist and Nature and has a stack of awards for his quality of writing and integrity. Before writing his article Yong even requested Bargh’s opinion but Bargh refused to give it. Thereby explaining why Yong’s critique may be ever so slightly superficial in this instance (Update 11/03/08: Yong has now published a full reply to Bargh’s blog containing a reply from a member of the team that created the replication).
Bargh’s argument is tainted from the offset with a rambling attack on the most revered, open access, not for profit academic journal (PLoS), claiming because the replication was published there that it is “essentially self-published”. PLoS have replied in the comments clearly proving this attack to be completely factually deficit. It’s worth noting that the open access PLoS is by no means alone in charging it’s authors. A great proportion of leading journals are “pay as you go publications” (as Bargh puts it) so this argument is invalid, not to mention weirdly juvenile coming from someone in Bargh’s position. The only difference is that other publishers that actually are profit making are just less up-front about charges, describing them as “colour” charges when there is often sod all printing going on. The Journal of Neuroscience for example charges $1000 per image (assuming you like your hard work portrayed in colour rather than appearing as something from the dark ages of Xerox). That’s on top of a $980 publication fee. By any standards PLoS is cheap relative to it’s competition, it even offers to waive the fee for any reason, no questions asked.
After you’ve waded through the school yard bullshit, if you’re still reading that is, the actual argument at hand is academic in the extreme (so I won’t paraphrase it here). If you take the time to read it, Bargh’s case is monumentally ironic. Besides the discussion on priming at hand, Bargh’s paper focuses on “rudeness” and “elderly stereotype”. It’s almost like Bargh is trying to parody himself as a rude old professor clinging on to a sinking ship, a casualty of post-publication review, open science and all the good things to come of it. You couldn’t make it up. If you do a search on Psych File Drawer you’ll discover this isn’t even the first failed replication attempt of Bargh’s experiment.
To me this debate highlights an issue at the very heart of science that is often neglected in Psychology. Science gets interesting when science gets replicated.
Doyen S, Klein O, Pichon CL, & Cleeremans A (2012). Behavioral priming: it’s all in the mind, but whose mind? PloS one, 7 (1) PMID: 22279526
Bargh, J. Chen, M. Burrows, L. (1996). Automaticity of Social Behaviour: Direct Effects of Trait Construct and Stereotype Activation on Action. Journal of Personality and Social Psychology : http://www.yale.edu/acmelab/articles/bargh_chen_burrows_1996.pdf
Subscribecontact directly by simply hitting reply to the email. You will never receive spam under any circumstances and you can unsubscribe at any time with one click. Alternately, use the link below to subscribe via RSS or your favourite reading platform.
Africa America Bad Science BCI Brain Computer Interfacing breaking news Cannabis Censorship Cocaine Copyright Counterfeit Drugs Daily Fail DailyFail daily mail Daily Mail Demolition Squad Drugs EEG Emotiv Fake Drugs FMRI Health Hoax Independent Misinformation Music Neuroscience Open Science Procrastination Psychology Rat Brain Robot Review Satire Science sex Skepticism Statistics Student Loans Crisis Susan Greenfield Synaesthesia Technology The confederacy of dunces Video walking War on Drugs Wikileaks