Contribution of statistical learning to mitigating the curse of dimensionality in reinforcement learning

Natural environments are abundant with patterns and regularities. It has been demonstrated that learning these regularities, especially through statistical learning, can greatly influence perception, memory, and other cognitive functions. Using a novel experimental paradigm involving two orthogonal tasks, we investigated whether regularities in the environment can enhance reward learning. In one task, human participants predicted the next stimulus in a sequence by recognizing regularities in a feature. In a separate, multidimensional learning task, they learned the predictive value of a different set of stimuli, based on reward feedback received after choosing between pairs of stimuli. Using both model-free and model-based approaches, we found that participants used regularities about features from the sequence-prediction task to bias their behavior in the learning task, resulting in the values associated with the regular feature having a greater influence. Fitting of choice behavior revealed that these effects were more consistent with attentional modulations of learning, rather than decision making, due to regularity manipulation. Specifically, the learning rates for the feature with regularity were higher, especially when learning from the forgone option during unrewarded trials. This demonstrates that feature regularities can intensify the confirmation bias observed in reward learning. Our results suggest that by enhancing learning about certain features, detecting regularities in the environment can reduce dimensionality and thus mitigate the curse of dimensionality in reward learning. Such interactions between statistical and reward learning have important implications for learning in naturalistic settings.

Damage explains function in spiking neural networks representing central pattern generator

Complex biological systems evolved to control dynamics in the presence of noisy and often unpredictable inputs. The staple example is locomotor control, which is vital for survival. Control of locomotion results from interactions between multiple systems--from passive dynamics of inverted pendulum governing body motion to coupled neural oscillators that integrate predictive forward and sensory feedback signals. The neural dynamic computations are expressed in the rhythmogenic spinal network termed the central pattern generator (CPG). While a system of ordinary differential equations or a rate model is typically "good enough" to describe the CPG function, the computations performed by thousands of neurons in vertebrates are poorly understood. To study the distributed computations of a well-defined neural dynamic system, we developed a CPG model for gait expressed with the spiking neural networks (SNN). The SNN-CPG model faithfully recreated the input-output relationship of the rate model, describing the modulation of gait phase characteristics. The degradation of distributed computation within elements of the SNN-CPG model was further studied with "lesion" experiments. We found that lesioning flexor or extensor elements, with otherwise identical structural organization of reciprocal networks, affected differently the overall CPG computation. This result mimics experimental observations. Moreover, the increasing general excitability within the network can compensate for the loss of function after progressive lesions. This observation may explain the response to spinal stimulation and propose a novel theoretical framework for degraded computations and their applications within restorative technologies.

A theory of temporal self-supervised learning in neocortical layers

The neocortex constructs an internal representation of the world, but the underlying circuitry and computational principles remain unclear. Inspired by self-supervised learning algorithms, we introduce a computational model wherein layer 2/3 (L2/3) learns to predict incoming sensory stimuli by comparing previous sensory inputs, relayed via layer 4, with current thalamic inputs arriving at layer 5 (L5). We demonstrate that our model accurately predicts sensory information in a contextual temporal task, and that its predictions are robust to noisy or partial sensory input. Additionally, our model generates layer-specific sparsity and latent representations, consistent with experimental observations. Next, using a sensorimotor task, we show that the model's L2/3 and L5 prediction errors mirror mismatch responses observed in awake, behaving mice. Finally, through manipulations, we offer testable predictions to unveil the computational roles of various cortical features. In summary, our findings suggest that the multi-layered neocortex empowers the brain with self-supervised learning capabilities.

Hierarchical cortical entrainment orchestrates the multisensory processing of biological motion

When observing others' behaviors, we continuously integrate their movements with the corresponding sounds to achieve efficient perception and develop adaptive responses. However, how human brains integrate these complex audiovisual cues based on their natural temporal correspondence remains unknown. Using electroencephalogram, we demonstrated that cortical oscillations entrained to hierarchical rhythmic structures in audiovisually congruent human walking movements and footstep sounds. Remarkably, the entrainment effects at different time scales exhibit distinct modes of multisensory integration, i.e., an additive integration effect at a basic-level integration window (step-cycle) and a super-additive multisensory enhancement at a higher-order temporal integration window (gait-cycle). Moreover, only the cortical tracking of higher-order rhythmic structures is specialized for the multisensory integration of human motion signals and correlates with individuals' autistic traits, suggesting its functional relevance to biological motion perception and social cognition. These findings unveil the multifaceted roles of entrained cortical activity in the multisensory perception of human motion, shedding light on how hierarchical cortical entrainment orchestrates the processing of complex, rhythmic stimuli in natural contexts.

The hippocampus pre-orders movements for skilled action sequences

Plasticity in the subcortical motor basal ganglia-thalamo-cerebellar network plays a key role in the acquisition and control of long-term memory for new procedural skills, from the formation of population trajectories controlling trained motor skills in the striatum to the adaptation of sensorimotor maps in the cerebellum. However, recent findings demonstrate the involvement of a wider cortical and subcortical brain network in the consolidation and control of well-trained actions, including an area traditionally associated with declarative memory - the hippocampus. Here, we probe which role these subcortical areas play in skilled motor sequence control, from sequence feature selection during planning to their integration during sequence execution. An fMRI dataset collected after participants learnt to produce four finger sequences entirely from memory with high accuracy over several days was examined for both changes in BOLD activity and their informational content in subcortical regions of interest. Although there was a widespread activity increase in effector-related striatal, thalamic and cerebellar regions, the associated activity did not contain information on the motor sequence identity. In contrast, hippocampal activity increased during planning and predicted the order of the upcoming sequence of movements. Our findings show that the hippocampus pre-orders movements for skilled action sequences, thus contributing to the higher-order control of skilled movements. These findings challenge the traditional taxonomy of episodic and procedural memory and carries implications for the rehabilitation of individuals with neurodegenerative disorders.

Models optimized for real-world tasks reveal the necessity of precise temporal coding in hearing

Neurons encode information in the timing of their spikes in addition to their firing rates. Spike timing is particularly precise in the auditory nerve, where action potentials phase lock to sound with sub- millisecond precision, but its behavioral relevance is uncertain. To investigate the role of this temporal coding, we optimized machine learning models to perform real-world hearing tasks with simulated cochlear input. We asked how precise auditory nerve spike timing needed to be to reproduce human behavior. Models with high-fidelity phase locking exhibited more human-like sound localization and speech perception than models without, consistent with an essential role in human hearing. Degrading phase locking produced task-dependent effects, revealing how the use of fine-grained temporal information reflects both ecological task demands and neural implementation constraints. The results link neural coding to perception and clarify conditions in which prostheses that fail to restore high-fidelity temporal coding could in principle restore near-normal hearing.

Anatomical circuits for flexible spatial mapping by single neurons in posterior parietal cortex

Primate lateral intraparietal area (LIP) is critical for cognitive processing. Its contribution to categorization and decision-making has been causally linked to neurons' spatial sensorimotor selectivity. We reveal the intrinsic anatomical circuits and neuronal responses within LIP that provide the substrate for this flexible generation of motor responses to sensory targets. Retrograde tracers delineate a loop between two distinct operational compartments, with a sensory-like, point-to-point projection from ventral to dorsal LIP and an asymmetric, more widespread projection in reverse. Neurophysiological recordings demonstrate that especially more ventral LIP neurons exhibit motor response fields that are spatially distinct from its sensory receptive field. The different associations of response and receptive fields in single neurons tile visual space. These anatomical circuits and neuronal responses provide the basis for the flexible allocation of attention and motor responses to salient or instructive visual input across the visual field.

Characterising time-on-task effects on oscillatory and aperiodic EEG components and their co-variation with visual task performance.

Fluctuations in oscillatory brain activity have been shown to co-occur with variations in task performance. More recently, part of these fluctuations has been attributed to long-term (>1hr) monotonous trends in the power and frequency of alpha oscillations (8-13 Hz). Here we tested whether these time-on-task changes in EEG activity are limited to activity in the alpha band and whether they are linked to task performance. Thirty-six participants performed 900 trials of a two-alternative forced choice visual discrimination task with confidence ratings. Pre- and post-stimulus spectral power (1-40Hz) and aperiodic (i.e., non-oscillatory) components were compared across blocks of the experimental session and tested for relationships with behavioural performance. We found that time-on-task effects on oscillatory EEG activity were primarily localised within the alpha band, with alpha power increasing and peak alpha frequency decreasing over time, even when controlling for aperiodic contributions. Aperiodic, broadband activity on the other hand did not show time-on-task effects in our data set. Importantly, time-on-task effects in alpha frequency and power explained variability in single-trial reaction times. Moreover, controlling for time-on-task effectively removed the relationships between alpha activity and reaction times. However, time-on-task effects did not affect other EEG signatures of behavioural performance, including post-stimulus predictors of single-trial decision confidence. Therefore, our results dissociate alpha-band brain-behaviour relationships that can be explained away by time-on-task from those that remain after accounting for it - thereby further specifying the potential functional roles of alpha in human visual perception.

Flexible integration of natural stimuli by auditory cortical neurons

Neurons have rich input-output functions for processing and combining their inputs. Although many experiments characterize these functions by directly activating synaptic inputs on dendrites in vitro, the integration of spatiotemporal inputs representing real-world stimuli is less well studied. Using ethologically relevant stimuli, we study neuronal integration in relation to Boolean AND and OR operations thought to be important for pattern recognition. We recorded single-unit responses in the mouse auditory cortex to pairs of ultrasonic mouse vocalization (USV) syllables. We observed a range of integration responses, spanning the sublinear to supralinear regimes, with many responses resembling the MAX-like function, an instantiation of the OR operation. Integration was more MAX-like for strongly activating features, and more AND-like for spectrally distinct inputs. Importantly, single neurons could implement more than one integration function, in contrast to artificial networks which typically fix activation functions across all units and inputs. To understand the mechanism underlying the flexibility and heterogeneity in neuronal integration, we modelled how dendritic properties could influence the integration of inputs with complex spectrotemporal structure. Our results link nonlinear integration in dendrites to single-neuron computations for pattern recognition.

A Libra in the Brain: Neural Correlates of Error Awareness Predict Moral Wrongness and Guilt Proneness

Error awareness is a fundamental mechanism in humans. Through traditional psychological tasks, certain neural activities that represent errors in objective manners have been identified. However, there is limited knowledge on how humans subjectively determine right from wrong in moral contexts. In this study, participants (N=39) mentally simulated themselves as the agents of moral and immoral behaviors, while viewing a series of actions with EEG recording and MRI scanning, respectively. A significant difference in error-related negativity (ERN) was observed among morally wrong scenarios, accompanied by higher wrongfulness ratings. Additionally, individual differences in guilt-proneness could predict the subjects' ERN amplitude. The ERN amplitude was correlated with the BOLD activity in the anterior mid-cingulate cortex and anterior insula to immoral scenarios, reflecting error awareness toward moral wrongfulness. The late potential component displayed greater negativity to immoral scenarios and was correlated with BOLD activities in the amygdala, ventromedial prefrontal cortex, and temporoparietal junction, indicating cognitive and affective evaluation in moral judgment. In line with the moral dynamic framework, our results demonstrated individual variability in moral judgments, as indicated by dispersed and overlapping cognitive neural networks. This suggests that subjective evaluations of wrongfulness are underpinned by neural mechanisms, associated with those involved in objective error awareness.