Last weekend I went for a walk in the woods. It’s not something I do often, but I was with friends who wanted to do it, so alright. We hiked for about four miles. It was not strenuous. It was warm, but not too warm, with a light breeze. The trees protected us from the sun. The canopy was intensely green, and the sky, peeking through, was cornflower blue.
While my friends walked along merrily, ostensibly enjoying their surroundings, I spent much of the hike wondering why anybody would bother with this kind of activity. I find hikes repetitious and boring. My mind goes idle (other than its constant scanning for beetles, ticks, rocks, and face-slapping branches). I look at my feet more the trees, and when I do look up, everything looks the same, a wash of brown and green. I try to think of it as good exercise, but the inefficiency! The same 90 minutes at the gym would be far more helpful.
It’s odd, when you think about it: My friends and I were all absorbing the same sights, sounds, smells and touches. We’re all about the same age and live in big cities. But they seemed to like it very much and I didn’t like it much at all.
Wilhelm Wundt, known by some as the father of psychology, pondered this very conundrum more than a century ago. In his 1896 book, Outlines of Psychology, Wundt wrote that our experiences can be broken into two elements: objective sensations and subjective affect. For instance, if you put your hand under hot water, you’ll feel a sensation of the heat. That’s the objective part. It’s hot, not cold. But then there’s the affect, the way the sensation affects you. For some people, the hot water will be excruciating, for others it will be rapturous. The sensation and the affect are inseparable, and both are crucial to perception, Wundt wrote: “The actual contents of psychical experience always consist of various combinations of sensational and affective elements.” Our experience, he added, “depends for the most part not on the nature of these elements so much as on their union.”
Wundt’s ideas have been difficult to prove empirically, but new evidence comes from a paper published Sunday in Nature Neuroscience. Adam Anderson, a cognitive psychologist at Cornell University, and his colleagues used brain scanners to show that our brains use different codes to represent these objective and subjective aspects of an experience.
In the first part of the experiment, the researchers showed 16 volunteers 128 pictures portraying a wide range of scenes—a girl checking a man’s pulse, a bloody surgery, an igloo, tree bark, three adorable puppies. The volunteers rated how positively or negatively they felt about each picture and the researchers compared these responses to volunteers’ brain activity while looking at each picture.
The researchers found that brain activity in the visual cortex, at the back of the head, tends to correspond with basic visual properties of a picture, such as its luminosity. Meanwhile, activity in a region called the ventral temporal cortex (VTC), above the ears, codes whether a picture contains inanimate or animate objects. And activity in the orbitofrontal cortices (OFC), just above the eyes, predicts how a participant rated a given picture — the affect code, you might say.
Intriguingly, the OFC seems to deal with both positive and negative ratings. That makes some sense, the researchers note, because monkey studies have pinpointed individual neurons in this region that respond to rewarding stimuli, aversive stimuli, or both. All three types of neurons are likely conflated in this human imaging study because a voxel represents the combined activity of hundreds of thousands of neurons.
The second part of the experiment investigated whether the brain uses the same affect code for different types of sensory experiences. “It is presently unknown,” the researchers explain,”whether the displeasure evoked by the sight of a rotting carcass and the taste of spoiled wine are at some level supported by a common neural code.”
So they put the same participants back in the scanner, but instead of showing them pictures they fed them liquids of different flavors: sour, sweet, bitter, salty, and no taste. Just as before, the volunteers rated each taste and the researchers had a look at how their brains responded.
It turns out that the brain has certain affect codes that are specific to the sensory experience. Brain activity in the VTC, for example, corresponded to the volunteers’ ratings of taste, but not their ratings of the pictures. However, their brains used other affect codes, in parts of the OFC as well as other regions, for both taste and sight ratings.
It’s not clear why the brain would have so many different affect codes, but it makes intuitive sense. If an experience had pleasant smells but frightening sounds, say, you’d want to be able to assess each one separately before making an overall judgment.
In the last part of the study, the researchers looked at these patterns across the 16 volunteers. They found that the volunteers gave different ratings for the same pictures and tastes — just as I realized in the woods, some people like things more than others do. But the way their brains coded positive ratings (and negative ratings) was similar for everybody, regardless of which stimuli were considered positive or negative.
Other researchers have used the same brain-imaging methods* to do “mind-reading” of other kinds — using brain scans to predict, for example, what pictures a person is looking at. The predictive accuracy is somewhat crude, for now, but good enough to be creepy. This new study adds another layer of creepiness, showing that even our most subjective personal experiences could be decoded by a machine.
To put it another way: If these researchers had appeared in the woods last weekend with a brain scanner, and convinced me to crawl into it and cogitate on my forest experience, their algorithms may have been able to predict how I was feeling about the hike. Of course, they could have just asked me. And what these algorithms can’t answer, not yet, is the most interesting question of all: What determines affect? Why oh why do people like the woods?
*A bit about the imaging methods, for the curious. Traditional brain imaging, as I blogged about a couple of weeks ago, looks at brain activity voxel by voxel. (A voxel is a three-dimensional pixel, or a cube.) Researchers ask whether the activity in a given voxel differs between two experimental conditions, and then average across all of the voxels that do show significant differences. One downside of this technique is that it won’t pick up voxels that show a weak response to a given stimulus because it will be impossible to tell whether the response was due to chance alone. This study, in contrast, used a method called multi-voxel pattern analysis, which analyzes brain activity across many voxels. It’s basically a way to pick up more fine-grained patterns of activity. The trade-off with this approach is that it’s more difficult to compare activity across individuals.