解码听觉表象与磁模式分析
English
Diviser
Vue d'ensemble
资料来源： 乔纳斯 · 卡普兰和莎拉一、 贝尔实验室 — — 南加利福尼亚大学
想象一下铃在响的声音。什么大脑的变化，当我们召唤这样的声音在”心灵的耳朵？”还有越来越多的证据，大脑的想象力，它用于感知使用相同的机制。1为例，当想象视觉图像，视觉皮层被激活，并时想象的声音，听觉皮层正在。然而，到何种程度上是这些感觉皮质的激活特定于内容的我们的想象力？
一种技术，可以帮助回答这个问题是磁模式分析 (MPVA) 的脑功能图像分析了使用机器学习技术。2-3中 MPVA 的实验中，我们训练一种机器学习算法来区分不同的刺激所诱发的活动的各种图案。例如，我们可能会问是否想象一个铃铛的声音产生不同图案的相比想象的电锯声或小提琴的声音的听觉皮层的活动。如果我们分类器学会分辨这些三刺激所产生的大脑活动模式，那么我们可以得出结论听觉皮层由每个刺激激活用独特的方法。认为这种实验的一种方法是，而不是活动的大脑区域，只询问一个问题我们问一个问题，关于这一地区的信息内容。
在这个实验中，基于迈耶等人，2010 年，4我们会提示与会者想象凭他们的无声视频，几种声音有可能唤起听觉表象。因为我们感兴趣测量诱发想象在听觉皮层中的细微模式，它要好，如果刺激介绍了完整的沉默，而不受干涉的功能磁共振成像扫描仪时发出的巨大声响。为了实现这一目标，我们将使用一种特殊的功能性磁共振成像序列被称为稀疏采样。在此方法中，单一功能磁共振成像卷每个刺激之后，定时捕获的血流动力学响应峰值是后天的 4-5 s。
Procédure
Résultats
The average classifier accuracy in the planum temporale across all 20 participants was 59%. According to the Wilcoxon Signed-Rank test, this is significantly different from chance level of 33%. The mean performance in the frontal pole mask was 32.5%, which is not greater than chance (Figure 2).
Figure 2. Classification performance in each participant. For three-way classification, chance performance is 33%. According to a permutation test, the alpha level of p < 0.05 corresponds to 42%.
The permutation test found that only 5% of the permutations achieved accuracy greater than 42%; thus, our statistical threshold for individual subjects is 42%. Nineteen of the 20 subjects had classifier performance significantly greater than chance using voxels from the planum temporale, while none had performance greater than chance using voxels from the frontal pole.
Thus, we are able to successfully predict from patterns of activity in auditory cortex which of the three sounds the participant was imagining. We were not able to make this prediction based on activity patterns from the frontal pole, suggesting that the information is not global throughout the brain.
Applications and Summary
MVPA is a useful tool for understanding how the brain represents information. Instead of considering the time-course of each voxel separately as in a traditional activation analysis, this technique considers patterns across many voxels at once, offering increased sensitivity compared with univariate techniques. Often a multivariate analysis uncovers differences where a univariate technique is not able to. In this case, we learned something about the mechanisms of mental imagery by probing the information content in a specific area of the brain, the auditory cortex. The content-specific nature of these activation patterns would be difficult to test with univariate approaches.
There are additional benefits that come from direction of inference in this kind of analysis. In MVPA we start with patterns of brain activity and attempt to infer something about the mental state of the participant. This kind of “brain-reading” approach can lead to the development of brain-computer interfaces, and may allow new opportunities for communication with those with impaired speech or movement.
References
- Kosslyn, S.M., Ganis, G. & Thompson, W.L. Neural foundations of imagery. Nat Rev Neurosci 2, 635-642 (2001).
- Haynes, J.D. & Rees, G. Decoding mental states from brain activity in humans. Nat Rev Neurosci 7, 523-534 (2006).
- Norman, K.A., Polyn, S.M., Detre, G.J. & Haxby, J.V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn Sci 10, 424-430 (2006).
- Meyer, K., et al. Predicting visual stimuli on the basis of activity in auditory cortices. Nat Neurosci 13, 667-668 (2010).
Transcription
Auditory imagery is a process that gives rise to the experience of hearing sounds, even when there are no external auditory stimuli present.
For instance, think about hearing the sound of a cell phone ringing. While information within memory underlies this imaginary event, evidence suggests that an individual’s brain uses the same mechanisms for imagination as those that are involved in the actual perception.
Just upon imagining the ringing, regions within the auditory cortex become activated. However, even though this is true across acoustic stimuli, how sounds are encoded to allow for the detailed processing of distinct sounds—like distinguishing between a doorbell chime and a song playing on the radio—is an important question.
Based on previous work by Meyer and colleagues, this video demonstrates how to combine functional magnetic resonance imaging—fMRI—with presentations of different silent videos to investigate how the brain responds to auditory imagery.
We will also describe how to use a method called multi-voxel pattern analysis—MVPA for short—to predict what subjects have imagined by analyzing patterns of activation obtained during fMRI sessions.
In this experiment, participants lie in an fMRI scanner and are shown a series of silent videos. Each one—whether it’s a rooster crowing, a chainsaw cutting through a tree, or a person playing a piano—evokes distinctive and vivid auditory imagery, and they are asked to imagine the sounds during every single presentation.
The imaging acquisition procedure relies upon sparse temporal sampling, whereby a single fMRI volume is acquired 4 to 5 s after each stimulus is presented. Such timing captures the peak of the hemodynamic response and reduces the likelihood that signals would be masked by scanner noise.
Each imagined sound is expected to induce subtle yet distinctive patterns of neural activity, specifically in the auditory cortex. Here, pattern is the key word: The classical way to analyze these data uses a univariate approach, where the individual voxels—representing some level of activation—are collapsed into a single average.
These values are then compared across sounds and may not produce any significant differences in activation levels.
Instead, using a multivariate analysis, multiple voxels are laid out for each sound and activation levels can be compared collectively, across all voxels—contributing to a unique overall pattern for each imagined sound.
With this multi-voxel pattern analysis, or MVPA, approach, if the patterns are indeed sensitive to specific content, then it’s possible that they could be used to predict the original stimulus. That’s right—MVPA is often referred to as a mind-reading technique!
To achieve this prediction aspect, more intensive processing must be performed after collecting the participants’ data, which are divided into training and testing sets.
Labeled data from the training set are first subjected to machine-learning calculations, specifically, a Support Vector Machine algorithm. This process is used to accurately classify data by recognizing features in the neural patterns that may distinguish the three types of sounds from one another.
After the classifier has learned features to accurately identify the types, it’s presented with unlabeled data from the testing set, and its guesses are then compared to the correct stimuli labels.
In this case, the classification performance serves as the dependent variable—recorded as the classifier’s accuracy—which is also compared to voxels evoked in a different location in the brain, such as the frontal pole.
The classifier is expected to predict the identification of auditory imagery, revealing the importance of MVPA in detecting content-specific activity within the auditory cortex.
Per experimental and safety concerns, verify that all participants are right-handed, have normal or corrected-to-normal vision, no history of neurological disorders or claustrophobia, and do not possess any metal in their body. Also, ensure that they fill out the necessary consent forms.
Before proceeding, explain that they will see several short silent videos in the scanner that may evoke a sound in their mind. Ask them to focus on the imagined sounds, “hear” them as best as they can, and to remain still for the duration of the task.
Now, prepare the participant to enter the scanner. To see these steps in detail, please refer to another fMRI video produced in this collection.
Following preparation, align the participant and send them inside the bore. In the adjacent room, first collect a high-resolution anatomical scan. Then, synchronize the start of the silent video presentation with the start of the functional scan.
To achieve sparse temporal sampling, set the acquisition time of an MRI volume to 2 s, with a 9-s delay in between.
Importantly, coordinate the start of each 5-s video clip to begin 4 s after the previous MRI acquisition starts to capture the hemodynamic activity that corresponds to the middle of the movie.
Present each video 10 times, in random order, generating one scanning session that lasts 5.5 min. Repeat this functional acquisition sequence three more times.
After the four functional scans have been performed, bring the participant out of the scanner, and debrief them to conclude the study.
To define regions of interest, use the high-resolution anatomical scans of each participant and trace voxels on the surface of the temporal lobe that correspond to the early auditory cortex, also known as the planum temporale. In addition, create a mask containing voxels in the frontal lobe, which will be used as the control region.
Then, preprocess the data by performing motion correction to reduce movement artifacts and temporal filtering to remove signal drifts.
Next, divide the data into two sets: training and testing. In one data set, train a classifier—a support vector machine algorithm—making sure to keep data from the two brain regions separate for each subject.
In the other set, assess what the classifier has learned—its ability to correctly guess the identity of unlabeled data—and record the algorithm’s accuracy across runs. Perform this procedure a total of four times, leaving out one functional scan as testing data each time—a process called cross-validation.
To visualize the data, graph the averaged classifier accuracies across the four cross-validation folds for each participant.
Plot these averages for both the primary region of interest—the planum temporale—and the control area—the frontal pole—to compare focal specificity of the classifier, the extent to which a particular area, such as the auditory cortex, is selectively predicted to be involved in auditory imagination.
In this case, run a non-parametric statistic, the Wilcoxon Signed-Rank test, to test performance against chance, which is 33%. Note that the average classifier accuracy in the auditory cortex was 59%, which is significantly different from chance level.
In contrast, the mean performance in the frontal pole mask was 33%, which is not significantly different from chance.
Moreover, notice that classifier performance varied across individuals. After using a permutation test to calculate a new statistical threshold of 42%, see that 19 of 20 subjects had accuracy values significantly greater than this level using voxels from the planum temporale, whereas none had performance greater than chance using voxels from the frontal pole.
Overall, these results imply that MVPA techniques accurately predicted which of the three sounds participants were imagining based on patterns of neural activity. Such predictions were only made within the auditory cortex, suggesting that acoustic content is not represented globally throughout the brain.
Now that you are familiar with how to apply multi-voxel pattern analysis to study auditory imagery, let’s look at how neuropsychologists use multivariate techniques to advance a futuristic approach to mind-reading—the decoding of mental states—in other domains.
Classifiers have been used on fMRI data obtained from the ventral temporal cortex to predict the kinds of objects participants viewed, distinguishing between houses and faces, for example.
Taking this a step further, it’s even possible to predict whether the individual would buy that house or find the person pleasant. As creepy as that sounds, these neuromarketing implications are not far-fetched!
The same approach could also be used to detect either emotional states after watching a show—recognizing that a scary film is indeed terrifying—or even the movie genre; for instance, the frightening flick might engage the amygdala more predictably over a contemplative one that would reliably involve the prefrontal cortex.
In addition, brain-computer interfaces could convert mental states into signals that would enhance communication, in the case of individuals undergoing speech therapy, or movements, for those who’ve suffered from amputation of a limb.
You’ve just watched JoVE’s video on understanding auditory imagery using multi-voxel pattern analysis. Now you should have a good understanding of how to design and conduct the auditory imagery experiment in conjunction with functional neuroimaging, and finally how to analyze and interpret specific patterns of brain activity.
Thanks for watching!