This dissertation advances research on evaluation (RoE) through a trio of studies focusing on the role of context and the innovative use of Linguistic Inquiry and Word Count (LIWC) software in formative evaluation in a qualitative research project. The first article extends Coryn et al. (2017) and elucidates how various contextual…

As language technologies have become more sophisticated and prevalent, there have been increasing concerns about bias in natural language processing (NLP). Such work often focuses on the effects of bias instead of sources. In contrast, this paper discusses how normative language assumptions and ideologies influence a range of automated language…

Despite the growing interest in AI for language assessment, there remains a significant research gap regarding its usefulness for assessing less proficient language skills, particularly those of learners of English as a second or foreign language (S/FL). AI models often prioritize proficient writing, neglecting the intricacies of learner language.…

Metadiscourse guides how readers interact with a text and process the information they find. Because texts differ in purpose and audience, so do patterns of metadiscourse use. This research examines the patterns of metadiscourse use in topic-based writing, developed following a structured authoring method. The resulting writing is modular,…

This experimental study, and partial replication of Sun and Hu (2020), examined how and to what extent direct (i.e., hands-on corpus use) or indirect (i.e., hands-off corpus use) online data-driven learning (DDL) modules were able to assist Chinese writers from an English as a foreign language (EFL) context in using more appropriate hedging (i.e.,…

Translators employ an array of tools to streamline the translation process, ensuring precision and coherence. The necessity of these tools is particularly pronounced when tackling authentic materials, such as texts from the United Nations (UN). Among these tools, corpora stand out as a pivotal resource. In light of this, this study undertook a…

Automatic essay scoring (AES) refers to the process of scoring free text responses to given prompts, considering human grader scores as the gold standard. Writing such essays is an essential component of many language and aptitude exams. Hence, AES became an active and established area of research, and there are many proprietary systems used in…

Although data collection has been used in language learning settings for some time, it is only in recent decades that large corpora have become available, along with efficient tools for their use. Advances in natural language processing (NLP) have enabled rich tagging and annotation of corpus data, essential for their effective use in language…

While studies exploring the overall effectiveness of Data Driven Learning activities have been positive, learner participants often seem to report difficulties in deciding what to look up, and how to formulate appropriate queries for a search (Gabel, 2001; Sun, 2003; Yeh, Liou, & Li, 2007). "The Prime Machine" (Jeaco, 2015) was…

The corpus-driven analysis of concordance data often results in the identification of groups of lines in which repeated patterns around the node item establish membership in a particular function meaning group (Mahlberg, 2005). This paper explains the KWICgrouper, a concept designed to support this kind of concordance analysis. Groups are defined…

WordSmith Tools, since its launch in 1996, has had a rather unusual history and the aim of this paper is to record some of the chief influences on its development. The paper thus presents and discusses the history of WordSmith Tools and its predecessors going back to the early 1980s when processors were much slower, memory very limited and disk…

Topic models like latent Dirichlet allocation (LDA) provide a framework for analyzing large datasets where observations are collected into groups. Although topic modeling has been fruitfully applied to problems social science, biology, and computer vision, it has been most widely used to model datasets where documents are modeled as exchangeable…

This paper describes a novel methodology for the detection of speech patterns. Lagged co-occurrence analysis (LCA) utilizes the likelihood that a target word will be uttered in a certain position after a trigger word. Using this methodology, it is possible to uncover a statistically significant repetitive temporal patterns of word use, compared to…

In this article we describe the emerging area of text classification research focused on the problem of collaborative learning process analysis both from a broad perspective and more specifically in terms of a publicly available tool set called TagHelper tools. Analyzing the variety of pedagogically valuable facets of learners' interactions is a…

Papers from the meeting on computational linguistics include: "Conversational Games, Belief Revision and Bayesian Networks" (Stephen G. Pulman); "Valence Alternation without Lexical Rules" (Gosse Bouma); "Filtering Left Dislocation Chains in Parsing Categorical Grammar" (Crit Cremers, Maarten Hijzelendoorn);…