This study is a conceptual replication of Teimouri et al.'s (2022) investigation into the validity of the second language (L2) grit scale (the L2-Grit scale). There are several concerns about the generalizability of the findings of Teimouri et al. (2022), especially regarding the discriminant validity of the scale and the relation of L2 grit with…

The purpose of this paper is to (a) establish whether meaning recall and meaning recognition item formats test psychometrically distinct constructs of vocabulary knowledge which measure separate skills, and, if so, (b) determine whether each construct possesses unique properties predictive of L2 reading proficiency. Factor analyses and…

Self-efficacy has emerged as a popular construct in second language research, especially in the frontline and practitioner-researcher spaces. A troubling trend in the relevant literature is that self-efficacy is often measured in a general or global manner. Such research ignores the fact that self-efficacy is a smaller context-driven construct…

The performance decision tree (PDT; Fulcher et al., 2011) is a rubric style that is applicable to performance assessment, with origins in Upshur and Turner's (1995) empirically derived binary-choice, boundary-definition (EBB) scale. It is easier for raters to assess performance by evaluating multiple binary-choice descriptors. Additionally,…

Addressing the lack of English tests that measure critical thinking (CT) abilities of EFL learners, this study aims to develop an English Critical Thinking Test (ECTT) designed to measure the consistency, analysis, and inference CT skills. It is a follow-up to a pilot study to accumulate validity evidence for the use of the ECTT. A total of 262…

Studies on Global Englishes (GE)-related pedagogical and curricular innovations emphasize the importance of exposing students to the diversity of English and its users. However, they do not provide direct evidence of changes in students' attitudes. The reliability and validity of existing quantitative instruments for measuring students' attitudes…

The purpose of this study is to determine whether a vocabulary size test is a valid and reliable measure of longitudinal growth in vocabulary size (VS), and to investigate the relationship between growth in VS and use of vocabulary learning strategies (VLS). Kasahara's (2006) revised version of Mochizuki's (1998) Vocabulary Size Test (VST) was…

The substantive component of construct validity requires a confrontation between empirical test results and content relevance. The Vocabulary Size Test (VST) has been extensively validated in terms of empirical results. Less is known, however, about expert judgments of content relevance. The VST was constructed and validated according to the…

Although an accurate measure of vocabulary size is integral to understanding the proficiency of language learners, the validity of multiple-choice (M/C) vocabulary tests to determine this has been questioned due to users guessing correct answers which inflates scores. In this paper the nature of guessing and partial knowledge used when taking the…

This study measures the perceptions of English language teachers at Japanese universities. It combines the results of online questionnaires on assessment practices and another one on teacher autonomy. Results suggest that according to teachers' self-reports of assessment practices, certain measures of assessment quality are being affected by the…

This study searched for open-source semantic similarity tools and evaluated their effectiveness in automated content scoring of fact-based essays written by English-as-a-Foreign-Language (EFL) learners. Fifty writing samples under a fact-based writing task from an academic English course in a Japanese university were collected and a gold standard…

'Grit' has been identified as an important predictor of success in a number of academic and non-academic domains (Duckworth, A. L., C. Peterson, M. D. Matthews, and D. R. Kelly. 2007. "Grit: Perseverance and Passion for Long-Term Goals." "Journal of Personality and Social Psychology" 92: 1087-1101.…

The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The "TOEFL® Essentials"™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the…

Background: Research on the test-taker perception of assessments has been conducted under the assumption that negative test-taker perception may influence test performance by decreasing test-taking motivation. This assumption, however, has not been verified in the field of language testing. Building on expectancy-value theory, this study explored…

In this study, we define the term "screener test," elaborate key considerations in test design, and describe how to incorporate the concepts of practicality and argument-based validation to drive an evaluation of screener tests for language assessment. A screener test is defined as a brief assessment designed to identify an examinee as a…