【论文笔记】Multivariate Encoding Analysis of Medial Prefrontal Cortex Cortical Activity during Task Learn

【论文笔记】Multivariate Encoding Analysis of Medial Prefrontal Cortex Cortical Activity during Task Learning

Abstract

1. 研究表明，内侧前额叶皮层（mPFC）负责结果评估。
2. 研究表明，mPFC在执行任务时可能在目标规划和行动执行中发挥重要作用。

如果mPFC中编码的信息能够被准确地提取和识别，那么它就可以通过更好地以奖励信息引导的方式重建受试者的运动意图来改进脑机接口的设计。

Linear-nonlinear-Poisson (LNP) model is applied for encoding analysis on mPFC neural spike data when a rat is learning a two-lever-press discrimination task. 采用线性-非线性-泊松（LNP）模型，在大鼠学习双杠杆按压辨别任务时，对mPFC神经峰值数据进行编码分析。

We use the $L^2$norm of tuning parameter in LNP model to indicate the importance of the encoded information and compare the spike train prediction performance of LNP model using all information, the most significant information and reward information only. 我们使用的是$L^2$化在LNP模型中调整参数的范数，以表明编码信息的重要性，并比较仅使用所有信息、最重要信息和奖励信息的LNP模型的峰值序列预测性能。

The preliminary results indicate that mPFC activity can encode simultaneously the information of goal planning, action execution and outcome evaluation and that all the relevant information could be reconstructed from mPFC spike trains on a single trial basis. 初步结果表明，mPFC活动可以同时编码目标规划、行动执行和结果评价等信息，所有相关信息都可以在单一试验基础上从mPFC峰值序列中重建。

Keywords

brain machine interface;

medial prefrontal cortex;

neural encoding;

I. INTRODUCTION

Existing BMIs were designed to execute pre-defined tasks and have difficulty in adapting to a new task. 现有的BMI被设计为执行预定义的任务，很难适应新的任务。

The medial prefrontal cortex (mPFC), especially the anterior cingulate cortex (ACC), is critically involved in reward-guided learning. 内侧前额叶皮层（mPFC），特别是前扣带皮层（ACC），在奖赏引导的学习中起着关键作用。

The predicted response outcome (PRO) model and reward value and prediction model (RVPM) addressed the role of mPFC in detecting discrepancies between actual and intended outcome. 预测反应结果（PRO）模型和奖励价值和预测模型（RVPM）探讨了mPFC在检测实际结果和预期结果之间的差异中的作用。

Holroyd et al. proposed that mPFC gives the goal and plan at the beginning of the task and monitors the movement execution during the task. mPFC在任务开始时给出目标和计划，并在任务期间监控任务的运动执行。

The mPFC integrates a variety of signals (including expected reward, costs, effort and so on) to determine whether, where and how much control to allocate during the task. Shenhav等人提出，mPFC集成了各种信号（包括预期奖励、成本、努力等），以确定在任务中是否、在哪里以及分配多少控制。

Shen et al. proposed an internally rewarded reinforcement learning-based BMI decoder which extracted reward information from mPFC spikes and guided the choice of the movement. Shen等人提出了一种基于内部奖励强化学习的BMI解码器，该解码器从mPFC峰值中提取奖励信息，并指导运动的选择。

X. Shen, X. Zhang, Y. Huang, S. Chen, and Y. Wang, “Reinforcement Learning based Decoding Using Internal Reward for Time Delayed Task in Brain Machine Interfaces,” no. 61836003, pp. 3351–3354, 2020.

We perform multivariate encoding analysis using rat data, while a male Sprague Dawley (SD) rat was trained to learn a two-lever-press discrimination task according to audio cues. Neural signals of 16 channels from mPFC were collected during the task learning. 我们使用大鼠的数据进行多变量编码分析，而一只雄性Sprague Dawley（SD）大鼠被训练根据音频线索学习双按辨别任务。在任务学习过程中，收集了mPFC中16个通道的神经信号。

Four kinds of information including start cue, movement preparation, movement execution and reward information are extracted from behavioral data and mapped into an eight-dimensional vector for encoding analysis. 从行为数据中提取开始线索、运动准备、运动执行和奖励信息等四种信息，并将其映射到一个八维向量中进行编码分析。

Kalman filter is applied to verify whether the kinematics related to movement preparation, movement execution and reward information can be reconstructed from mPFC spike trains on a single trial basis. 应用卡尔曼滤波验证了运动准备、运动执行和奖励信息是否可以重建与运动准备、运动执行和奖励信息相关的运动学。

II. METHOD

A. Behavioral Experiment Design and Data Preprocessing

实验过程：

1. Six male Sprague Dawley (SD) rats were first well trained on a one-lever-press task and then started to learn to perform a two-lever-press discrimination task. 先执行one-lever-press任务，再执行two-lever-press任务。
2. Each trial of the task was initialized by an audio cue (lasting $900ms$) of either a high pitch ($10kHz$) or low pitch ($1.5kHz$), which was randomly generated. 持续900毫秒的高频噪声和低频噪声
3. The rat needed to press the high lever when hearing the high-pitched cue and press the low lever when hearing the low-pitched cue. 大鼠需要按压高杠杆当听到高频声音，反之亦然
4. If the lever was correctly pressed within $5s$ after the start cue and held for $500ms$, a feedback cue (lasting $90ms$) with the same pitch would be presented and the subject would be rewarded with a water drop. 任务执行正确了，又喝水奖励；
5. Wrong pressing, early releasing and omission all led to an unsuccessful trial and the rat would neither hear the feedback cue nor get water reward. The intertrial interval was set to be a random value ranging from 3 to 6s. 任务忽略或提早释放了，没有奖励

The offline sorter (Plexon Inc, Dallas, Texas) was utilized to sort the single neuron from each channel and the spike timing information was restored. 利用离线分类器对每个通道中的单个神经元进行分类，并恢复尖峰时间信息。

Meanwhile, all the behavior events and their timings including the trial start cue presenting, lever pressing, lever releasing, feedback cue presenting were recorded by the behavior recording system (Lafayette Instrument, USA) and synchronized with the aforementioned neural recording system. 同时，所有行为事件及其时间，包括试验开始提示、杠杆按压、杠杆释放、反馈提示呈现均由行为记录系统（美国拉斐特仪器）记录，并与上述神经记录系统同步。

Fig. 1 shows the mapping results of four kinds of information. For each kind of information, we use a two-dimensional variable to describe how it changes over time. 图1为四种信息的映射结果。对于每一种信息，我们使用一个二维变量来描述它如何随时间的变化。

B. Multivariate Encoding Analysis of mPFC spike trains

Compared with other encoding models which use linear, exponential or Gaussian tuning function, LNP model builds the tuning characteristic of each single neuron without any prior assumption on the tuning properties. 与其他使用线性、指数或高斯调优函数的编码模型相比，LNP模型构建了每个单个神经元的调优特性，而无需对调优特性进行任何预先假设。

After estimating the two distributions from the training set and calculating the nonlinear function for every neuron, we can obtain the instantaneous firing rate ${\lambda }_t$ of the Poisson model from the output of $f(y)$ and finally establish a mapping from the multi-dimensional information vector to the spike trains of mPFC. 从训练集估计两个分布并计算每个神经元的非线性函数，我们可以得到瞬时放电率${\lambda }_t$的泊松模型的输出为$f(y)$最后建立了从多维信息向量到mPFC脉冲序列的映射。

C. Reconstruct kinematics and reward information

We divide the original data into training set (75%) and testing set (25%). The training data is used to obtain the parameters of the Kalman filter, including state transition matrix, measurement matrix and noise covariance matrix. 我们将原始数据划分为训练集（75%）和测试集（25%）。利用训练数据获得卡尔曼滤波器的参数，包括状态转移矩阵、测量矩阵和噪声协方差矩阵。

Here, the correlation coefficient is calculated to evaluate the decoding performance by comparing reconstruction states with the ground truth. 在这里，通过比较与地面真实值的重建状态，计算相关系数来评估解码性能。

III. RESULT

After sorting the mutual information of all 18 mPFC neurons by descending order, we choose the top 10 neurons for encoding analysis. 我们对所有18个mPFC神经元的互信息按降序排序后，选择前10个神经元进行编码分析。

We can see that the firing rates predicted by LNP model with all information are the closest to the ground truth. 我们可以看到，LNP模型预测的发射率都是最接近地面真相的.

The prediction performance with the most significant information is worse than that with all information and better than that with reward information only. 使用最重要信息的预测性能比使用所有信息的预测性能差，而优于仅使用奖励信息的预测性能。

We can observe only blue line keeps in the confidence range, which demonstrate that the prediction from LNP model with all information is closer to the real observation than other two predictions. 我们只能观察到蓝线保持在置信范围内，这表明具有所有信息的LNP模型的预测比其他两种预测更接近真实观测。

All the above results, including correlation coefficient values and KS-test, demonstrate that mPFC activity may encode information for goal planning, action execution and outcome evaluation simultaneously. 以上所有结果，包括相关系数值和ks检验，表明mPFC活动可以同时编码目标规划、行动执行和结果评估的信息。