Integrating neural and ocular attention reorienting signals in virtual reality

Objective. Reorienting is central to how humans direct attention to different stimuli in their environment. Previous studies typically employ well-controlled paradigms with limited eye and head movements to study the neural and physiological processes underlying attention reorienting. Here, we aim to better understand the relationship between gaze and attention reorienting using a naturalistic virtual reality (VR)-based target detection paradigm. Approach. Subjects were navigated through a city and instructed to count the number of targets that appeared on the street. Subjects performed the task in a fixed condition with no head movement and in a free condition where head movements were allowed. Electroencephalography (EEG), gaze and pupil data were collected. To investigate how neural and physiological reorienting signals are distributed across different gaze events, we used hierarchical discriminant component analysis (HDCA) to identify EEG and pupil-based discr...

A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface

Objective. Brain-computer interface (BCI) aims to establish communication paths between the brain processes and external devices. Different methods have been used to extract human intentions from electroencephalography (EEG) recordings. Those based on motor imagery (MI) seem to have a great potential for future applications. These approaches rely on the extraction of EEG distinctive patterns during imagined movements. Techniques able to extract patterns from raw signals represent an important target for BCI as they do not need labor-intensive data pre-processing. Approach. We propose a new approach based on a 10-layer one-dimensional convolution neural network (1D-CNN) to classify five brain states (four MI classes plus a ‘baseline’ class) using a data augmentation algorithm and a limited number of EEG channels. In addition, we present a transfer learning method used to extract critical features from the EEG group dataset and then to customize the model to the sin...

Neural tracking to go: auditory attention decoding and saliency detection with mobile EEG

Objective. Neuro-steered assistive technologies have been suggested to offer a major advancement in future devices like neuro-steered hearing aids. Auditory attention decoding (AAD) methods would in that case allow for identification of an attended speaker within complex auditory environments, exclusively from neural data. Decoding the attended speaker using neural information has so far only been done in controlled laboratory settings. Yet, it is known that ever-present factors like distraction and movement are reflected in the neural signal parameters related to attention. Approach. Thus, in the current study we applied a two-competing speaker paradigm to investigate performance of a commonly applied electroencephalography-based AAD model outside of the laboratory during leisure walking and distraction. Unique environmental sounds were added to the auditory scene and served as distractor events. Main results . The current study shows, for the first time, t...

A state-informed stimulation approach with real-time estimation of the instantaneous phase of neural oscillations by a Kalman filter

Objective. We propose a novel method to estimate the instantaneous oscillatory phase to implement a real-time system for state-informed sensory stimulation in electroencephalography (EEG) experiments. Approach. The method uses Kalman filter-based prediction to estimate current and future EEG signals. We tested the performance of our method in a real-time situation. Main results. Our method showed higher accuracy in predicting the EEG phase than the conventional autoregressive (AR) model-based method. Significance. A Kalman filter allows us to easily estimate the instantaneous phase of EEG oscillations based on the automatically estimated AR model implemented in a real-time signal processing machine. The proposed method has a potential for versatile applications targeting the modulation of EEG phase dynamics and the plasticity of brain networks in relation to perceptual or cognitive functions.

Improving reaching with functional electrical stimulation by incorporating stiffness modulation

Objective. Intracortical recordings have now been combined with functional electrical stimulation (FES) of arm/hand muscles to demonstrate restoration of upper-limb function after spinal cord injury. However, for each desired limb position decoded from the brain, there are multiple combinations of muscle stimulation levels that can produce that position. The objective of this simulation study is to explore how modulating the amount of coactivation of antagonist muscles during FES can impact reaching performance and energy usage. Stiffening the limb by cocontracting antagonist muscles makes the limb more resistant to perturbation. Minimizing cocontraction saves energy and reduces fatigue. Approach. Prior demonstrations of reaching via FES used a fixed empirically-derived lookup table for each joint that defined the muscle stimulation levels that would position the limb at the desired joint angle decoded from the brain at each timestep. This study expands on that pr...

EEG-based brain–computer interfaces exploiting steady-state somatosensory-evoked potentials: a literature review

A brain–computer interface (BCI) aims to derive commands from the user’s brain activity in order to relay them to an external device. To do so, it can either detect a spontaneous change in the mental state, in the so-called ‘active’ BCIs, or a transient or sustained change in the brain response to an external stimulation, in ‘reactive’ BCIs. In the latter, external stimuli are perceived by the user through a sensory channel, usually sight or hearing. When the stimulation is sustained and periodical, the brain response reaches an oscillatory steady-state that can be detected rather easily. We focus our attention on electroencephalography-based BCIs (EEG-based BCI) in which a periodical signal, either mechanical or electrical, stimulates the user skin. This type of stimulus elicits a steady-state response of the somatosensory system that can be detected in the recorded EEG. The oscillatory and phase-locked voltage component characterising this response is called a steady-state som...

Improve P300-speller performance by online tuning stimulus onset asynchrony (SOA)

Objective . The P300-Speller is a classic brain–computer interface paradigm that has been subjected to numerous clinical trials. Some studies have reported that the performance of the P300-Speller is closely related to stimulus onset asynchrony (SOA), but very few studies have attempted to improve the performance of the P300-Speller by optimizing SOA. Approach. In this paper, we designed a P300-Speller system based on a variable SOA and dynamic stop strategy, which can automatically adjust SOA according to real-time operational performance. Main results. The online experiment results of 18 subjects showed that the event-related potential classifier and the dynamic stop algorithm established at 200 ms SOA can maintain the performance at a certain level among 50–300 ms SOA. The system can then reduce the SOA from an initial 200 ms to an average of about 98.5 ms while maintaining letter output accuracy. The average theoretical information transfer rate was sign...

Gastric neurons in the nucleus tractus solitarius are selective to the orientation of gastric electrical stimulation

Objective. Gastric electrical stimulation (GES) is a bioelectric intervention for gastroparesis, obesity, and other functional gastrointestinal disorders. In a potential mechanism of action, GES activates the nerve endings of vagal afferent neurons and induces the vago-vagal reflex through the nucleus tractus solitarius (NTS) in the brainstem. However, it is unclear where and how to stimulate in order to optimize the vagal afferent responses. Approach. To address this question with electrophysiology in rats, we applied mild electrical currents to two serosal targets on the distal forestomach with dense distributions of vagal intramuscular arrays (IMAs) that innervated the circular and longitudinal smooth muscle layers. During stimulation, we recorded single and multi-unit responses from gastric neurons in NTS and evaluated how the recorded responses depended on the stimulus orientation and amplitude. Main results. We found that NTS responses were highly sel...

Wireless microelectrode arrays for selective and chronically stable peripheral nerve stimulation for hindlimb movement

Objective . Maximizing the stability of implanted neural interfaces will be critical to developing effective treatments for neurological and neuromuscular disorders. Our research aims to develop a stable neural interface using wireless communication and intrafascicular microelectrodes to provide highly selective stimulation of neural tissue. Approach . We implanted a wireless floating microelectrode array into the left sciatic nerve of six rats. Over a 38 week implantation period, we recorded stimulation thresholds and movements evoked at each implanted electrode. We also tracked each animal’s response to sensory stimuli and performance on two different walking tasks. Main results . Presence of the microelectrode array inside the sciatic nerve did not cause any obvious motor or sensory deficits in the hindlimb. Visible movement in the hindlimb was evoked by stimulating the sciatic nerve with currents as low as 4.1 µ A. Thresholds for most of the 96 elect...

Effect of visual input on syllable parsing in a computational model of a neural microcircuit for speech processing

Objective. Seeing a person talking can help us understand them, particularly in a noisy environment. However, how the brain integrates the visual information with the auditory signal to enhance speech comprehension remains poorly understood. Approach. Here we address this question in a computational model of a cortical microcircuit for speech processing. The model consists of an excitatory and an inhibitory neural population that together create oscillations in the theta frequency range. When stimulated with speech, the theta rhythm becomes entrained to the onsets of syllables, such that the onsets can be inferred from the network activity. We investigate how well the obtained syllable parsing performs when different types of visual stimuli are added. In particular, we consider currents related to the rate of syllables as well as currents related to the mouth-opening area of the talking faces. Main results. We find that currents that target the excitatory n...