On Reading Minds

Nature¹

Who hasn't had the desire to see through another's eyes? Some researchers at Berkeley think they've taken the first steps towards achieving such a goal.

Jack L. Gallant and his lab-mates have managed the feat of decoding human fMRI measurements in such a way that they can infer the image that generated the recorded neuronal activity¹. fMRI as a technique assesses brain excitation indirectly, through blood-flow. The degree of excitation is clearly in some way related to the BOLD (Blood-oxygen-level dependent) signal obtained, but it is a bit crude in the sense that it isn't very spatially or temporally precise². The data can pinpoint activity to a few square millimeters, and within a window of about 6 seconds.

The paper detailing their results, appearing in Nature, describes how this remarkable trick was accomplished. First, the researchers consulted fMRI signals from subjects viewing a wide variety of natural images. They correlated this information with the pixels in the pictures themselves, and this allowed them to construct a model which predicted the pattern of blood-flow one might observe with fMRI in response to an arbitrary image. Once this was done, they essentially turned the model on it's head so that they could ascertain the viewed image from the fMRI data. In fact, at present it's quite a brute force approach that requires that the scientist have a set of images which are fed into the model to generate synthetic fMRI data to compare with the measured signals. However, it is possible that models of this form will eventually be sophisticated enough to avoid this.

If these techniques could, for example, be extended to other forms of mental reckoning, we might some day be able to see into the thoughts of those who are unable to communicate. Regardless of the practical applications, and however far from sneaking a peek the richly textured visual experience we each have, this type of savvy utilization of data and modeling techniques is exciting because it tickles the basic desire we all have to know another's being.

Notes/References:

1. Kay KN, Naselaris T, Prenger RJ, Gallant JL. (2008) Identifying natural images from human brain activity. Nature, 452;352-355
2. Other technologies sacrifice the large volume of brain space that fMRI can cover for spatial precision (over 10000x better, single cells) and temporal precision (over 100000 times better, though that much is not necessary).