We investigate whether people might come to produce utterances that they regard as ungrammatical by examining the production of ungrammatical verb-construction combinations (e.g., "The dancer donates the soldier the apple") after exposure to both grammatical and ungrammatical sentences. We contrast two accounts of how such production might take place: an abstract structural persistence account, according to which it is caused by increased activation of an abstract structural rule; and a lexically-driven persistence account, according to which it requires previous exposure to the same (ungrammatical) verb-construction combination. In four structural priming experiments, we found that sentences with ungrammatical verb-construction combinations were produced only after exposure to similar ungrammatical exemplars containing the same verb, but not after such sentences with a different verb, or grammatical sentences with the same construction. These results indicate that people can produce sentences with ungrammatical verb-construction combinations after brief exposure to related sentences, and provide support for the lexically-driven persistence account of such production. (Contains 9 tables and 3 figures.)