This study investigated to what extent the scores of two English tests are correlated to each other, namely, the English test of the "Common Test for University Admissions" (Common Test, henceforth) in Japan and the "TOEIC Bridge", a commercially available English test developed by "Educational Testing Service" (ETS)…

Language testing research has suggested that placement accuracy can be an important factor in the consequences of placement decision-making. However, previous studies primarily have used a one-shot teacher judgment of placement accuracy, suffering from a lack of students' perspectives of their time-varying experiences. The present study aimed to…

The performance decision tree (PDT; Fulcher et al., 2011) is a rubric style that is applicable to performance assessment, with origins in Upshur and Turner's (1995) empirically derived binary-choice, boundary-definition (EBB) scale. It is easier for raters to assess performance by evaluating multiple binary-choice descriptors. Additionally,…

Addressing the lack of English tests that measure critical thinking (CT) abilities of EFL learners, this study aims to develop an English Critical Thinking Test (ECTT) designed to measure the consistency, analysis, and inference CT skills. It is a follow-up to a pilot study to accumulate validity evidence for the use of the ECTT. A total of 262…

This study explored two assessment approaches to oral performances: analytical complexity, accuracy, and fluency (CAF) indices and human raters' evaluations. CAF indices are frequently used in second-language speaking (L2) research; however, because tasks are communicative and goal-oriented, the degree to which students achieve such communicative…

Background: Variations in the skill profiles of learners have become an important research area in recent years. However, there is a lack of empirical research on this topic in Japan. We conducted three studies to address this gap. Methods: Study 1 investigated the characteristics of the flat and uneven skill profiles of Japanese learners of…

Background: Research on the test-taker perception of assessments has been conducted under the assumption that negative test-taker perception may influence test performance by decreasing test-taking motivation. This assumption, however, has not been verified in the field of language testing. Building on expectancy-value theory, this study explored…

Background: In our research project, we have developed a scoring rubric for a second language (L2) summary writing for English as a foreign language (EFL) students in Japanese universities. This study aimed to examine the applicability of our five-dimensional rubric, which features both analytic and holistic assessments, to classrooms in the EFL…

Background: This study aims to develop a low-stakes assessment tool to establish a classroom English language benchmark that Japanese teachers of English can use for their own professional development purposes. To start with, we describe the differences between CLA (Classroom Language Assessment) in Hong Kong and the IDS (Integrative Diagnostic…

Background: English language writing is important in university entrance examinations in Japan and influences what learners and teachers do in English classes. This is especially noticeable in secondary education in Japan. While much research on university entrance examinations in Japan has been conducted, little attention has been paid to the…

This study was designed to determine how well existing analytic rating scales functioned in the assessment of low- to mid-proficiency Japanese university students' interactive English speaking ability when engaged in small group discussions. Many-facet Rasch measurement (MFRM) was employed to evaluate the quality of adapted rating scales for…

Background: The Common European Framework of Reference (CEFR) developed by Council of Europe has been gaining popularity since its publication in 2001, and reshaping the practice in language teaching, learning and assessment not only across the European continent but also in many other places such as Japan (CEFR-J project). Some pioneering work…

The National Center Test is a test that high school graduates take in order to be matriculated into the majority of universities in Japan. Its English section (NCT), though, is scheduled to be replaced by commercially available private English tests of four skills (reading, listening, speaking, writing) due to the fact that NCT currently measures…

Background: This study investigates how the criterion-referenced (CR) approach has impacted relationships between goals, classroom practices, and student achievement in English education in Japan from 1994 to the present, a period covering three government-mandated administrations of a national curriculum (Course of Study). No study has…

Background: While numerous articles on Criterion® have been published and its validity evidence has accumulated, test users need to obtain relevant validity evidence for their local context and develop their own validity argument. This paper aims to provide validity evidence for the interpretation and use of Criterion® for assessing second…