To build an Artificial Intelligence system that can assist us in daily lives, the ability to understand the world around us through visual input is essential. Prior studies train visual perception models by defining concept vocabularies and annotate data against the fixed vocabulary. It is hard to define a comprehensive set of everything, and thus…

Speech in everyday conversations is riddled with discourse markers (DMs), such as "well," "you know," and "like." However, in many lab-based studies of speech comprehension, such DMs are typically absent from the carefully articulated and highly controlled speech stimuli. As such, little is known about how these DMs…

In real-world settings, vision language models (VLMs) should robustly handle naturalistic, noisy visual content as well as domain-specific language and concepts. For example, K-12 educators using digital learning platforms may need to examine and provide feedback across many images of students' math work. To assess the potential of VLMs to support…

There is now a well-established literature showing that people anticipate upcoming concepts and words during language processing. Commonsense knowledge about typical event sequences and verbal selectional preferences can contribute to anticipating what will be mentioned next. We here investigate how temporal discourse connectives…

A key benefit in intelligent Computer Assisted Language Learning (iCALL) is that it allows complex linguistic phenomena to be incorporated into digital learning platforms, either for the autonomous learner or to complement classroom teaching. The present paper describes (1) complex phonological/ morphophonemic alternations of Irish, which are…

Research into typing patterns has broad applications in both psycholinguistics and biometrics (i.e., improving security of computer access via each user's unique typing patterns). We present a new software package, TypingSuite, which can be used for presenting visual and auditory stimuli, collecting typing data, and summarizing and analyzing the…

Describes a graphic modeling, natural language processing system, VerbalImage, which mimics features of human shape reasoning. Subjects read the same text description and were able to recognize the image generated by the computer from among a series of other computer-generated images. Performance on task was as good as for a control group…

Examines the vocabulary and communication constructs that are used by novices and domain experts to describe objects in an object identification task. Results suggest that indexing and retrieval systems should provide semantic level similarity mechanisms to allow for whole-object as well as part-wise visual analogy. The systems should also provide…

Explores the indirect method of locating for indexing the likely explicit and implicit captions of photographs, using multimodal clues including the specific words used, syntax, surrounding layout of the Web page, and general appearance of the associated image. The MARIE-3 system thus avoids full image processing and full natural-language…