Study raises questions about the role of brain scans in courtrooms

A murder suspect sits in a quiet room with electrodes placed on her head. The prosecution reads out its narrative of the crime and the suspect’s alleged role in it. As she listens, the machines record her brain activity and reveal that she experienced aspects of the crime that only the murderer could have. Her own memories, teased out by technology, have betrayed her. The verdict is guilty.

This scenario might seem far-fetched, but it actually happened in an Indian trial that took place in 2008. The judge took a brain scan as proof that the suspect had “experiential knowledge” about the crime that only the killer could have possessed. She was sentenced to life in prison. There has been a smattering of attempts to use brain-scanning technology in this way, accompanied by an uproar about the technology’s readiness.

Now, a new study by Jesse Rissman from Stanford University confirms that promises about the social implications of brain scans are overplayed. Together with Henry Greely and Anthony Wagner, Rissman has shown that brain scans can accurately decode whether people think they remember something, but not whether they actually remember something. And that gap between subjective and objective memory is a vast chasm as far as the legal system is concerned.

Our memories are stored within networks of neurons so it’s reasonable to think that by studying the patterns of activity within these networks, we should be able to decipher individual memories. Studies have already started to show that this is possible with our existing brain-scanning techniques, and with every positive result, the temptation to use such advances in a practical setting grows.

The courtroom is an obvious candidate, especially because our brains respond differently when it experiences something new compared to something old. You could use brain scans to tell if someone has actually seen a place, person or thing, reliably corroborating the accounts of witnesses and suspects without having to rely on the vagaries of accurate recall and moral fortitude. For this reason, techniques like functional magnetic resonance imaging (fMRI) have been enticingly billed as the ultimate in lie detection technology. Claims of “mind-reading machines” and “psychic computers” have abounded in the press.

To assess these claims, Rissman asked 16 volunteers who studied 210 faces for four seconds each. An hour later, they saw 400 faces, half of which were old and half were new. They had to tell the two groups apart. For each face, they had one of five responses – a certain recollection, or a sense that it was old or new, with either high or low confidence. Throughout these trials, Rissman scanned the volunteers’ brains using fMRI. He used pattern recognition software to analyse the scans and identify patterns of activity that corresponded to each response – a “neural signature” of viewing an old face, or a new one.

At first, his software seemed to be very accurate. When Rissman analysed trials where the recruits correctly classified the faces, his software could separate the old and new faces based on brain activity alone, with an average accuracy of 83%. In trials where the volunteers were most confident in their judgments, that score rose as high as 95%.

The software passed other challenges too. For trials when the faces were all old (or all new), it separated ‘hits’ (where the volunteers rightly said that they’d seen the faces before) from ‘misses’ (where they incorrectly said that the faces were new) with an accuracy of 75%. It could separate trials where the recruits felt certain or confident in their judgments from those where they weren’t so confident with accuracies between 79 and 90%. And best of all, the software could even reliably decode the brain scans of one individual after it was “trained” on the data from another.

So far, everything seemed promising. But all of these tests focused on the recruits’ subjective memories – what they thought they remembered.  If fMRI scans are to be truly admissible in court, they have to do better than that. Scientists must be able to use them to decode a person’s objective memories – whether they actually remember something they saw.

To assess that, Rissman focused on trials where the recruits classified a face as old with low confidence. He wanted to see if the software could tell the difference between faces that were actually old and correctly remembered, from those that were actually new and falsely remembered. It succeeded, but only just, with an average accuracy of 59%. For trials where they classified faces as new with low confidence, the program did even worse at telling the right assessments from the wrong ones – guesswork would have been just as good.

These results were bolstered by a variation on the same experiment. Again, Rissman showed seven recruits a set of 210 faces but this time, he told them to rate their attractiveness rather than memorise them. When they saw the larger set of 400 faces, they initially only had to say if they were male or female. The brain scans should still be able to reveal whether they recognised the faces even when they aren’t explicitly trying to do so.  But they couldn’t – it only achieved an accuracy of 56%, not significantly different from guesswork.

These results are impressive and disappointing at the same time. They demonstrate that fMRI can decode the neural signatures of subjective recognition, at least under controlled laboratory conditions. It also shows that software trained on one person can be used to reliably decode the brain activity of another – that’s fascinating in itself because it suggests that these neural signatures are highly consistent from person to person. Vaughan Bell from King’s College London (and the excellent Mind Hacks blog) says, “In other words, it is identifying brain activation patterns for conscious experiences of remembering, which seem to generalise across people.”

But given all that, the technique’s inability to separate what people think they saw from what they actually did limits its use as a source of legal evidence. It means that the scans are only as good as the memories of the people who are being scanned, and we know that our memories are fickle and sometimes untrustworthy things.

Bell says, “Any potential fMRI ‘lie detector’ technology may be equally as liable to the memory distortions that affect eye witness testimony or other forms of courtroom recall. Perhaps a little speculatively, this may mean that although such technology could pick up someone who deliberately lies about what they remember, it may not be able to distinguish between someone whose memory had become distorted over time or who has come to believe false information.”

Of course, fMRI scans have already found their way into courtrooms and more attempts are on the horizon. Just last week, a Brooklyn judge dismissed fMRI evidence from an employer-retaliation case, and Wired reports that on May 13, a Tennessee court will hear arguments over the admissibility of fMRI evidence in another hearing. Both cases involved a company called Cephos (whose CEO has, incidentally, turned up on this blog before).

But Rissman’s work casts serious doubts over the role of suggests that there are massive barriers to the use of fMRI scans in court. Of course, unreliable evidence such as eyewitness testimony is often used in trials, but the big danger for brain scans is their appearance of reliabilityappearance of reliability. What could be more compelling than a view inside someone’s mind? And what could be more dangerous than an unreliable source of evidence that is over-interpreted as being reliable, as the recent Indian case attests to?

Rissman concludes his paper with a stark warning. He says, “The neuroscientific and legal communities must maintain an ongoing dialogue so that any future real-world applications will be based on, and limited by, controlled scientific evaluations that are well understood by the legal system before their use. Although false positives and false negatives can have important implications for memory theory, their consequences can be much more serious within a legal context.”

Reference: PNAS http://dx.doi.org/10.1073/pnas.100102810 If this link isn’t working, read why here

Image by http://www.flickr.com/photos/killermonkeys/304439098/

More on fMRI scans:

• Your brain on Oprah and Saddam (and what that says about Halle Berry and your grandmother)
• Creating God in one’s own image
• Will vs. Grace – are people honest because they resist temptation or because they don’t feel it?
• Pre-emptive blood flow raises big questions about fMRI
• Computer predicts brain activity associated with different objects

//