From speech to pictures—how the brain develops language acquisition and comprehension skills

We take much about language for granted. And yet, perhaps more than any other characteristic it defines us as human beings. Language provides us a means of saying who and what we are, and serves as our primary vehicle for interacting with others. Brown has had a longstanding tradition of studying the nature of human language. Researchers in Brown’s Brain Science Program (BSP) investigate language processing from a number of perspectives –

the infant to the young child acquiring the language system,
the adult using the language system for speaking and understanding,
the language-impaired adult manifesting deficits in the processing mechanisms underlying language communication,
the neural basis of language, and
the computational and mathematical properties of language

The BSP faculty who participate in these and other approaches to language include Sheila Blumstein, Eugene Charniak, Katherine Demuth, William Heindel, Mark Johnson, Pauline Jacobson, Philip Lieberman, James Morgan, Julie Sedivy, and David Sobel.

The Acquisition of Language

Recognizing spoken words in real time is a formidable task, but performed effortlessly by the brain. Because speech is evanescent, the time allotted to identify which of the thousands of words known to the listener has just been heard must be measured in milliseconds. For infants, who must acquire the necessary strategies and capacities even as they are struggling to comprehend what they are hearing, the task seems especially daunting. Yet infants succeed in acquiring these abilities over the normal course of the first year or so of development.

Studying how infants acquire these skills is an interdisciplinary endeavor, involving computational or neural network modeling as well as an array of experimental techniques. At Brown, researchers are investigating how infants segment possible words from continuous speech, locating their likely beginning and ending points in the speech stream; how infants represent the acoustically rich input in a form that is amenable for efficient word recognition, learning to ignore or attenuate irrelevant dimensions of the signal; and how infants identify word-instances as exemplars of known word types, so that related semantic and syntactic knowledge can be accessed. Infants’ perceptual representations of words help to shape their own productions, from babbling to sentences, another research focus within Brown’s Brain Science Program. Naturally, the content of what infants and children say is linked to their level of cognitive development. Scientists at Brown are inquiring into the nature of this linkage.

Language Processing in Adults

In processing language, we select the appropriate word from our mental lexicon by mapping the incoming acoustic signal on to lexicon form. However, any individual word in our mental lexicon has many potential competitors—words that are similar in sound shape and/or words that are similar in meaning. How do we select the appropriate lexical entry from these competitors? When we speak, our productions vary considerably from moment to moment. How do these variations affect the mapping from sound structure on to lexical form and ultimately the selection of the appropriate word? BSP researchers at Brown explore these questions using an integrated, interdisciplinary approach and a variety of behavioral methodologies, including eye tracking with normal adults and brain-injured patients including aphasics, Parkinson and Alzheimer patients. Together with functional neuroimaging studies of normal subjects, Brown scientists are seeking to identify the processing mechanisms and the neural systems underlying speech and lexical processing.

Computational and Mathematical Linguistics

Computational linguistics investigates, in computational terms, how information is manipulated and transformed during language understanding, production and learning. Computational models allow us to quantify the steps involved in these linguistic processes, and permit us to mimic these processes on a computer. This field is highly interdisciplinary, and involves faculty and students from the Cognitive and Linguistic Sciences, Computer Science and Applied Mathematics departments. In the last decade there has been a statistical revolution in computational linguistics. These days most of our models are statistically based, and trained from large corpora consisting of many millions of words of text or speech.

One of the things that makes language so interesting is that it incorporates a rich internal structure that is largely hidden, i.e., not overtly marked in the words of a sentence. Much of the research in Brown’s BSP focuses on learning models of language which capture this rich internal structure in one way or another, and applying these models to problems such as language understanding, machine translation (the automatic translation of a document from one language to another) and information extraction (identifying who did what to whom in a document).




Posted 11/03