Idiomatic Language: Multiword Expressions

Stanford University, Stanford CA

Investigators

Abstract

With National Science Foundation support, Dr. Ivan Sag will conduct three years of linguistic research on multiword expressions (MWEs). MWEs are a problem that must be solved on the route to robust and natural natural language technology. Fixed expressions can be entered into a lexicon as words-with-spaces, but this is inadequate even for cases like 'part(s) of speech' and 'kicks/kick/kicked/kicking the bucket' ("die"). Further, some semi-fixed expressions exhibit positional variation (e.g., 'look up the answer/look the answer up'), while others of equal semantic idiosyncrasy do not (e.g., 'falling off a log/*falling a log off'). Some patterns appear rule-governed, but contain unexpected exceptions (e.g., 'call/ring/phone/*telephone someone up'). Decomposable phrasal idioms allow even greater syntactic flexibility: 'Kim pulled the strings that got Pat the job' and 'The strings that had been pulled in order for Pat to get the job were more extraordinary than those pulled to get Chris employed'. Linguistic research has not yet provided an adequate theory of the diverse phenomena that populate the space between truly fixed expressions and syntactically flexible phrasal idioms. This project will fill that gap, developing a mathematically precise and computationally tractable theory of various classes of MWEs. The research team will analyze and computationally manipulate corpus data, integrating discrete and frequentistic methodologies into a hybrid theory of the different kinds of MWEs. The methods include complex word structures, lexical selection, partially similar grammar rules organized into "construction hierarchies", idiomatic construction rules - a new technique for analyzing constructions where idiomatic expressions may be separated from one another by considerable distance. Compositionally structured, institutionalized phrases like 'traffic light' and 'phone booth' will be treated as purely statistical dependencies. The most recent extraction techniques will be used to develop stochastic constraints and integrate them into fundamentally discrete, constraint-based grammars. Since there is at present no comprehensive account of MWEs, this research on MWEs will contribute to both basic grammatical theory and our understanding of lexical knowledge. Because the analyses are mathematically precise and implemented within open-source software, the results will be of immediate utility for the development of robust language processing technology in a variety of constraint-based frameworks currently being explored in the field. Natural language processing applications will benefit from the results, including those involving language understanding, language generation, machine translation, and speech-related systems of various kinds (including speech prostheses for individuals with certain disabilities). All such applications involve scaling grammars up; and scaling grammars up to deal with MWEs will necessitate finding the right balance among various analytic techniques. Of special importance will be finding the right balance between symbolic and statistical techniques, a difficult problem whose solution this project's results bear on.