The ability to read someone’s mind has traditionally been the stuff of fiction. Our thoughts and experiences are private to us and we can choose when we share them with others. But with developments in brain scanning technology, mind-reading is becoming a hard science rather than a false promise. You may no longer need to be superhuman to see the darkest thoughts and desires of the person opposite you. You can instead convince them to lie in your brain scanner.
A basic test for a mind-reading machine is to tell you what visual image you are holding in your head. If the machine has to decide which of two images you are thinking of, does it perform significantly better than guessing at random?
In some cases, this is a fairly easy task with a brain scanner. For example, if you are trying to guess whether someone is thinking of playing tennis or walking around their house, you find different areas of the brain that are most active: the supplementary motor area for the motor imagery involved in playing tennis, and the parahippocampal place area for the spatial imagery involved in walking around your house. This distinction has been used to communicate with hospital patients who lie motionless in what appears to be a coma. If patients can use a thought to answer “yes” or “no” to questions (e.g. thinking of tennis for yes and house for no), then the doctor knows that the patient is showing signs of consciousness.
When you have a larger number of visual images to choose from or greater similarity between images, the task of decoding what someone is thinking becomes far more difficult. The overall levels of activity across the brain might be very similar for seeing a leopard versus a duck, so you need to be more sophisticated in how you analyze brain imaging data. One option is to drill down into detailed patterns of activity within a single area.
You can start scientific mind-reading by decomposing a list of images into their different visual features (e.g. object position, orientation, light contrast, etc). Then, you can take a set of practice images and train a decoder machine to link the features for those images with patterns of activity in the visual areas of a person’s brain as they see those features. Each feature drives brain activity in a different direction, so every unique combination of features corresponds to a unique pattern of activity overall.
After the machine is trained, you show the person brand new images they’ve never seen before and measure the patterns of activity in their visual brain areas. By using the associations that the decoder picked up during training, you can infer the visual features for the new image they see from their brain activity patterns. The machine can then look through a database of images, and estimate which image the person is seeing based on how closely the inferred features from brain activity match the actual decomposed features of an image. The closest match becomes the machine’s best guess.
The amazing thing is that because there is so much overlap in how our brain responds to actual visual images and the visuals we imagine in our head, you can do the same thing for what people are thinking about. Instead of showing them a visual image in the brain scanner, you just ask them to visualize a particular object in their mind. By analyzing brain activity in the same way, the machine can correctly infer which object they are imagining, or even which piece of famous artwork they have in mind.
One of the most exciting applications for this kind of mind-reading may be in decoding the content of our dreams while we sleep. Dreams are not only notoriously difficult to understand, they are often so vague and disconnected with reality that we barely remember them when we wake up. However, as with the imagined versus seen images I mentioned above, there is strong overlap in our visual brain patterns corresponding to seen images and dreamt images.
Researchers put participants in a brain scanner, waited until they fell asleep, and then woke them up during the most dream-intensive phase of sleep. By asking them to describe any visual images they saw while asleep, the researchers built a record of the images that people dreamed and their brain activity during those moments. By training a decoder machine on brain activity when people physically saw different images while awake, they could successfully read and predict what people visualized in their sleep from the same patterns of brain activity. As this kind of technology develops and improves, we should end up with more accurate and more comprehensive dream-reading machines.
Memory is another important function that depends on our ability to generate mental images. Long-term memory is similar to the type of visualizing I described in the experiments above. If I ask you to imagine a leopard or your tennis swing, you are recalling elements from your long-term memory of past experiences with those images or actions. But you also have working memory, which refers to your capacity to hold a number of objects in mind for seconds or minutes while doing a task. You may be trying to hold a phone number in mind while you dial it, or perhaps pictures during a memory game.
Visual areas of the brain generally do not sustain their overall level of activity when we hold visual images in memory. But as I explained before, you may need to drill down to find patterns of brain activity that code for specific images. This is exactly what one group of researchers tested when they asked participants to remember the orientation of a quickly flashed visual object for 11 seconds. After that delay period, participants had to decide whether a new comparison object was the same as the object in their memory. The decoder could analyze activity in visual areas of the brain during that delay period, and guess which of two orientations people were holding in their memory with over 80% accuracy. So even though brain activity in those areas returns to its resting level after seeing a visual object, it continues to exhibit a pattern of activity matching that object as you hold it in memory. Those same patterns also reliably predict the image if you generate it yourself in your mind instead of holding it in working memory over a delay.
The activity in our brain is naturally responsible for our mental experiences. Decoding those experiences with brain scanners is a radical and enlightening new project. We have already hit successes in decoding the contents of our mental imagery, dreams, and working memory. It’s thrilling to consider where we go from here. Although it’s easy to worry about future misuse of this technology (e.g. with invasions of privacy), the scientific journey itself could realistically improve how we understand people’s conscious experiences and potentially how we treat mental health disorders. In essence, it could teach us about the most important facts in our lives: where our thoughts come from, what they do to us, and how we can change them for the better.