< Back to previous page

Project

Deep neural networks as a model of speech perception

Developing and individualising hearing aids is challenging because no complete computational models of the auditory system exist, and no objective methods exist to differentially diagnose disorders along the auditory pathway. We aim to develop a computational model of the auditory system, based on state-of-the art, deep-neural-network-based systems for automatic speech recognition. The model will be constructed by letting people listen to natural speech signals, and relating the recorded electroencephalogram (EEG) signal to the corresponding acoustic signals. We will extend the latest results in encoder-decoder deep learning architectures and adapt them for application on EEG signals. We will (1) first build a system to predict the EEG signal from sound input. We will measure the degree to which the speech signal is coded in the brain at different stages, ranging from the basic acoustic features, to phoneme, word, and sentence-level representations. We then (2) move on to the reverse problem: decode speech at various representation stages from EEG recordings. Finally (3), we validate the deep neural network as a model for human speech recognition. The results will have applications in neuro-steered hearing aids, and diagnostics of speech and language disorders. We will establish a framework for application of deep learning to decoding of EEG signals  and collect substantial relevant data sets. The integration of cutting edge deep learning techniques to brain decoders may also have applications in brain-computer interfaces, leading to robust systems able to directly read and act on people's thoughts.
Date:1 Oct 2018  →  Today
Keywords:speech perception, automatic speech recognition, deep learning, auditory neuroscience, EEG
Disciplines:Applied mathematics in specific fields, Computer architecture and networks, Distributed computing, Information sciences, Information systems, Programming languages, Scientific computing, Theoretical computer science, Visual computing, Other information and computing sciences, Modelling, Biological system engineering, Signal processing, Multimedia processing, Neurosciences, Biological and physiological psychology, Cognitive science and intelligent systems, Developmental psychology and ageing