Reconstructing Speech Digitally from Human Auditory Cortex

Scientists believe the first step has been taken towards hearing imagined speech using a form of electronic telepathy. They hope that, in the future, it may be possible to “decode” the thoughts of brain-damaged patients who cannot speak.

The human brain has evolved in such a way that it decodes highly variable acoustic sounds into meaningful elements of language such as phonemes and words. Scanning these brain functions has always been difficult, since invasive recording of cortical activity is usually not possible. PLos Biology have undertaken a study to measure directly from the cortical surface, to characterize how the human brain performs speech recognition. Using these recordings, they are trying to determine what speech sounds can be decoded and reconstructed digitally.

The team employed a computer model that helped map out which parts of the brain were firing at what rate, when different frequencies of sound were played. With the help of that model, when patients were presented with words to think about, the team was able to guess which word the participants had chosen. They were even able to reconstruct some of the words, turning the brain waves they saw back into sound on the basis of what the computer model suggested those waves meant.

“The human auditory system deconstructs speech and other complex sounds into elementary time-frequency representations prior to higher level phonetic and lexical processing. This early auditory analysis, proceeding from the cochlea to the primary auditory cortex, yields a faithful representation of the spectro-temporal properties of the sound waveform, including those acoustic cues relevant for speech perception, such as formants, formant transitions, and syllable rate. However, relatively little is known about what specific features of natural speech are represented in intermediate and higher order human auditory cortex.”

It sounds like science fiction – maybe one day a mad scientist might remove a person’s brain from the body, suspend it in a vat of life-sustaining liquid, and connect its neurons by wires to a supercomputer which it could speak through.

Sources [PLos Biology] [BBC]