The fundamental problem faced by automatic text understanding in Natural Language Processing (NLP) is to identify semantically related pieces of text and integrate them together to compute the meaning of the whole text. However, the principle of compositionality runs into trouble very quickly when real language is examined with its frequent…

The past two decades have shown an unexpected effectiveness of "Web-scale" data in natural language processing. Even the simplest models, when paired with unprecedented amounts of unstructured and unlabeled Web data, have been shown to outperform sophisticated ones. It has been argued that the effectiveness of Web-scale data has…

Producing a sentence involves encoding a preverbal message into a grammatical structure by retrieving lexical items and integrating them into a functional (semantic-to-grammatical) structure. Individuals with agrammatism are impaired in this grammatical encoding process. However, it is unclear what aspect of grammatical encoding is impaired and…