Though reports of pedagogic materials production point to the range of compromises authors make when writing language teaching textbooks, many accounts are retrospective in nature. This study sought to expand the research perspective by interrogating writing episodes via qualitative content analysis to discover how two expert ELT (English language…

This study explores how machine translation (MT) influences the English writing process and performance of 29 9th-grade EFL students with limited English proficiency. Over 10 writing tasks conducted during the semester, participants had varied accessibility to MT. The research compared their performance when MT was permitted versus when it was…

It is widely believed that human rating performance is influenced by an array of different factors. Among these, rater-related variables such as experience, language background, perceptions, and attitudes have been mentioned. One of the important rater-related factors is the way the raters interact with the rating scales. In particular, how raters…

This paper investigates what matters to medical domain experts when setting standards on a language for specific purposes (LSP) English proficiency test: the Occupational English Test's (OET) writing sub-test. The study explores what standard-setting participants value when making performance judgements about test candidates' writing responses,…

This study investigates how implicit and explicit learning and knowledge are associated, by focusing on the salience of target form--meaning connections. The participants were engaged in incidental learning of artificial determiner systems that included grammatical rules of [± plural] (a taught rule), [± actor] (a more salient hidden rule), and [±…

In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater…

This study examines the decision-making behaviors of raters with varying levels of experience while assessing EFL essays of distinct qualities. The data were collected from 28 raters with varying levels of rating experience and working at the English language departments of different universities in Turkey. Using a 10-point analytic rubric, each…

This study investigates grammar learning strategies and the acquisition of L2 tense-aspect morphology through the retrospective think-aloud processes of a group of L3 learners of Catalan. A total of 18 students of Catalan in a university setting in Catalonia were asked to select one of three tense-aspect forms (preterite, imperfect, and perfect)…

Some metacognitive mental activities were explored on chemical reading activities as specific judgments called metacognitive judgments. A think-aloud protocol was carried out to fifteen chemistry teacher candidates to express their judgments on chemical reading activities for question posing. Two chemical articles were designed to stimulate these…

Think-aloud protocols (TAPs) are frequently used in research on essay rating processes. However, there are very few empirical studies of the completeness of TAP data and the effects of this technique on rater performance (i.e., rating processes and outcomes). This study aims to start to address this research gap. As part of a larger study on rater…

Various factors contribute to variability in English as a second language (ESL) essay scores and rating processes. Most previous research, however, has focused on score variability in relation to task, rater, and essay characteristics. A few studies have examined variability in essay rating processes. The current study used think-aloud protocols…

This dissertation tests a grammatical structure, differential object marking (DOM), which is particularly difficult for L2 learners to acquire. DOM is a phenomenon in which some direct objects are morphologically marked to distinguish them from subjects (Comrie, 1979). In Spanish, animate and specific direct objects are marked with the preposition…

Approaching peer review from a process and contextualized perspective, this exploratory case study investigates two Chinese EFL learners' decision-making patterns while evaluating peers' texts in an online peer review and factors influencing these patterns. Detailed qualitative case study data were collected through think-aloud protocols,…

Students' interpretations of their academic writing tasks has been a central concern in the cognitive-based writing research due to the prominent role such decision-making plays in determining students' subsequent thinking and composing strategies and ultimately in shaping their textual quality. Without a comprehensive understanding of how L2…

Educators often have to choose among different types of rating scales to assess second-language (L2) writing performance. There is little research, however, on how different rating scales affect rater performance. This study employed a mixed-method approach to investigate the effects of two different rating scales on EFL essay scores, rating…