One of my all time favourite bloggers, Oxford Neuroscientist Prof. Dorothy Bishop, or DeevyBee as she is known on Twitter has performed an amazing open access lecture focusing largely on the misunderstanding of neuroscience (click down to the “Emanuel Miller Lecture” to play the video). The talk is incredibly informative and digestible, even those with no understanding of neuroscience or psychology whatsoever will take a great deal away. The problem of the poor understanding of neuroscience is one of the main reasons why I started this blog, so if you like this blog then you’ll love this lecture.
The talk begins with a reasoned explanation of how and when we should be sceptical of neuroscience research, Bishop goes on to cite 4 key reasons why certain kinds of scientific research will inevitbaly result in false-positives:
“The four horseman of the apocalypse”
1. Maturation – People develop naturally over time.
“There seems to be an implicit assumption that the brain, because it is a physical organ is somehow not going to change unless you give it some intervention – that it is there as a static thing. This is completely untrue… as evidenced by this series of images.”
2. Practice effects – when people keep doing the same test again and again, they get better at it.
“…purely to do with the fact that you have got better at doing the test and nothing to do with your abilities… People forget that this can apply to language tests and thing like that. It also applies to some extent to the brain, often we don’t know how important this is because brain imaging is so new.. clearly if you get brain responses to novelty, that means if you do something twice – the first time round you will get different responses to the second time round when it is no longer novel”.
3. Regression to the mean - a statistical artefact of longitudinal studies that is exacerbated if you select participants on the basis of a low score on a test (for example participants with developmental difficulties). Bishop does an outstanding job of explaining the problem at about 18 minutes in to the talk.
“Regression to the mean is as inevitable as death and taxes”
Campbell and Kenny (1999) A primer on regression artefacts
4. The placebo effect. This is the obvious consideration that continues to impact poorly designed research but according to Bishop, the three issues listed above could actually be having an even greater impact than the placebo effect.
Bishop explains that a control group is vital in order to achieve valid findings, but a control group alone is not enough, we should also be asking questions such as:
- Are the groups randomly assigned – or is there some other factor at play?
- Is the control group given an alternative treatment? If not, why not?
- What causes drop out? People don’t tend to drop out at random and this can have a very big effect on results.
Weisberg, Deena Skolnick. (2008). The Seductive Allure of Neuroscience Explanations. Journal of Cognitive Neuroscience, 18 (3), 229-477 DOI: 10.1162/jocn.2008.20040
Campbell and Kenny (1999) A primer on regression artefacts
Today TED is an insatiable kingpin of international meme laundering—a place where ideas, regardless of their quality, go to seek celebrity, to live in the form of videos, tweets, and now e-books. In the world of TED—or, to use their argot, in the TED “ecosystem”—books become talks, talks become memes, memes become projects, projects become talks, talks become books—and so it goes ad infinitum in the sizzling Stakhanovite cycle of memetics, until any shade of depth or nuance disappears into the virtual void. Richard Dawkins, the father of memetics, should be very proud. Perhaps he can explain how “ideas worth spreading” become “ideas no footnotes can support.”
…The TED ideal of thought is the ideal of the “takeaway”—the shrinkage of thought for people too busy to think.
Zimbardo has served as the president of the American Psychological Association and has written a number of bestsellers and textbooks… and then this distinguished psychologist came to TED and delivered a rapid-fire bombardment of disconnected statistics and sweeping generalizations without any serious evidence backing them up. In this talk, he ends with a warning that our species will descend to the level of banana slugs. It’s like the punchline of a joke.
I’m not a fan of over-generalising, and as rampant over-generalising is one of my biggest criticisms of TED, I’m going to try not to do the same thing here. I’ve seen some tremendous TED talks and I’ve seen some horrendous TED talks. TED has it’s problems but TED is only a platform, though a powerful platform it may be. It’s not a source in itself and it’s clearly not where people should be headed for their sole education – there are far more promising on-line destinations for this than TED, but there again – TED told us that already.
If I were to be in charge of TED, I would not necessarily ask for higher quality-control, I’d ask to see better referencing, I certainly don’t want to see TED reacting to criticism by policing what speakers can say. As long as sources are adequately provided, I’m happy to do the leg work and fact-check myself, or let the hive-mind do it for me – but this becomes a nightmarish task when you can’t find a source to start with. Another thing I’d like to see is questions from the audience. I’ve heard people say time and time again that the most interesting part of lectures is the Q&A. This has certainly been my experience and I think it’s sad that TED does not allow its audience the freedom to question the speakers at the pulpit of ideas. As Zimmer so eloquently put it, “When a TED talk ends, the lights go out. There’s no time for questions”.
For all its faults, TED remains a great springboard for ideas and inspiration. I’m not convinced that its style is really as bigger problem as has been made out by some, it serves (or at least it should serve) a very different purpose to true on-line learning. I see TED much the same way as I see Twitter, never as a source – but often as a doorway to discovery. I do however agree that we need to change the attitude that TED is somehow a stamp of approval for fact-checked information, that it is not. The attitude we should have to TED is no different to the attitude we should have to all things – a healthy scepticism. This TED talk (ironically), sums up both this attitude and the purpose of TED (as I perceive it) just perfectly:
It’s an inspiring talk, it’s from someone from a strongly academic background but who is not chiefly an academic. It’s not particularly well referenced but that’s sort of the point. It’s not meant to be a lecture, it’s meant to kick-start you in to thinking for yourself. A little inspiration can go a very long way, but that doesn’t mean we shouldn’t always check, question and challenge our sources. Always, always, always. Even.. no.. especially when, our source is TED.
A new documentary discusses a bizarre affliction that has been widely (and uncritically) reported in the media to affect around a dozen of the approximately one million Japanese tourists who visit Paris each year. Paris Syndrome is said to occur when a combination of factors leave tourists with a particularly severe case of culture shock.
Symptoms are purported to include:
‘acute delusional states, hallucinations, feelings of persecution (perceptions of being a victim of prejudice, aggression, or hostility from others), derealization, depersonalization, anxiety, and also psychosomatic manifestations such as dizziness, tachycardia (and) sweating’ - Wikipedia
Due to the relatively microscopic numbers reported, it seems to me to be all too likely that Paris has become the unfortunate victim of an illusory correlation. After all, twelve out of one million is a number that doesn’t immediately strike me as particularly statistically significant, to say the least. To put this figure in to perspective, seven per thousand of the adult population are expected to suffer a schizophrenic episode at some point in their life. That is the equivalent of seven thousand per million, a number which suddenly makes twelve per million sound much smaller indeed.
Unfortunately, there is little by the way of well documented reports or case studies of Paris Syndrome and Professor Hiroaki Ota, the author of the original report1 published in a French psychiatric magazine, appears somewhat impossible to contact. It is certainly stirring to entertain the thought that perhaps something as seemingly benign (to a modern western generation) as a holiday to Paris, could spark a breakdown so severe it requires the victim to seek refuge and be accompanied back to their home country. In many ways there is some logic to the idea, there has certainly been plenty of theorising by the media and the internet’s hivemind. In reality however, the explanation for this peculiar condition may come down to chance as much as it does to the cultural relationship between Parisians and their Japanese guests.
1. Viala, A., Ota, H., Vacheron, M.N., Martin, P., & Caroli, F. (2004). “Les Japonais en voyage pathologique à Paris : un modèle original de prise en charge transculturelle”. Neuvure de journal Psychiatrie, 5, 31-34
NB: I couldn’t figure out how to access the article referenced by the documentary, if anyone has access and the time and inclination to translate it from French I’d greatly appreciate it.
The line below is taken from my undergrad revision notes, it is the same conclusion reached in a profile of Thomas Kuhn posted yesterday on the Scientific American blog.
“Kuhn described what science tends to be, Popper described what it ought to be”
Thomas Kuhn and Karl Popper were two philosophers with opposing views of the nature of science. Popper believed that the only way for science to progress is through testing falsifiable ideas, this theory has become the bedrock of scientific progress. Kuhn argued that this isn’t how science works in practice – according to Kuhn, scientists in reality tend to spend most of their time “mopping up”, which means producing results that fit in with the established view point. Kuhn popularised the term “paradigm shift”, used to describe the event that occurs when a new discovery blows previously held beliefs out of the water, for example when the theory of evolution won the argument over theories based around intelligent design. I’ve always struggled to see why the two theories (Kuhn’s and Poppers’) are so often seen as mutually exclusive, from my perspective both theories are indeed correct and in fact complement each other to serve important lessons in their own right. Testing falsifiable theories is clearly the way forward but at the very same time this must be complemented by replicating results, if this does not happen then we have no way of knowing that we are on the right track and this is a very bad thing for science.
The Scientific American piece adds an interesting angle to the Kuhn-Popper debate, both philosophers are now deceased but the piece describes a fascinating account of how Thomas Kuhn really felt about how his writings were percieved. The piece reveals Kuhn’s sentiment that he was “much fonder of (his) critics than (his) fans”, this will ring true for many that have written about science and felt the frustration of having their views misunderstood. For Kuhn it seems, the frustration really got to him. He was left isolated, frustrated by students who misunderstood his message and fearing to speak to journalists who he felt would only misrepresent his viewpoint. The SciAm piece adds an often neglected aspect to the debate, perhaps “scientists can never truly understand the ‘real world’ or even each other”, the way forward is to balance a healthy scepticism with a genuine openness to new ideas. This argument, was made in a fantastic essay by the late Carl Sagan:
“It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas. Obviously those two modes of thought are in some tension. But if you are able to exercise only one of these modes, whichever one it is, you’re in deep trouble.
If you are only skeptical, then no new ideas make it through to you. You never learn anything new. You become a crotchety old person convinced that nonsense is ruling the world. (There is, of course, much data to support you.) But every now and then, maybe once in a hundred cases, a new idea turns out to be on the mark, valid and wonderful. If you are too much in the habit of being skeptical about everything, you are going to miss or resent it, and either way you will be standing in the way of understanding and progress.
On the other hand, if you are open to the point of gullibility and have not an ounce of skeptical sense in you, then you cannot distinguish the useful as from the worthless ones. If all ideas have equal validity then you are lost, because then, it seems to me, no ideas have any validity at all.
Some ideas are better than others. The machinery for distinguishing them is an essential tool in dealing with the world and especially in dealing with the future. And it is precisely the mix of these two modes of thought that is central to the success of science.”
I’ll leave this discussion there and implore you to go ahead and read John Horgan’s SciAm piece and Carl Sagan’s essay as well as Ed Yong’s Nature Magazine piece, the debates complement each other perfectly leaving a great deal worth pondering.
Horgan, J. (2012). What Thomas Kuhn Really Thought about Scientific “Truth” Scientific American, Cross-Check
Sagan, C. (1987). The Burden of Skepticism Skeptical Inquirer, vol. 12
Yong E (2012). Replication studies: Bad copy. Nature, 485 (7398), 298-300 PMID: 22596136
Subscribecontact directly by simply hitting reply to the email. You will never receive spam under any circumstances and you can unsubscribe at any time with one click. Alternately, use the link below to subscribe via RSS or your favourite reading platform.
Africa America Bad Science BCI Brain Computer Interfacing breaking news Cannabis Censorship Cocaine Copyright Counterfeit Drugs Daily Fail DailyFail daily mail Daily Mail Demolition Squad Drugs EEG Emotiv Fake Drugs FMRI Health Hoax Independent Misinformation Music Neuroscience Open Science Procrastination Psychology Rat Brain Robot Review Satire Science sex Skepticism Statistics Student Loans Crisis Susan Greenfield Synaesthesia Technology The confederacy of dunces Video walking War on Drugs Wikileaks