Concerns have recently been raised about the validity of scales used in the L2 motivational self system tradition, particularly in relation to sufficient discriminant validity among some of its scales. These concerns highlight the need to systematically examine the validity of scales used in this tradition. In this study, we therefore compiled a…

This paper addresses the intersection of testing and policy, situating test-driven impact and validation within the context of policy-led educational reform in Korea. I will briefly review the existing validation models. Then, arguing for an expansion of the conventional conceptualization of consequential validity research, I use Fairclough's…

This study develops and validates an English communicative competence model for Korean high school students, in response to the need to redefine the relevant concepts and components of competence that are demanded by the rapidly evolving future society. Drawing on Celce-Murcia's (2008) theoretical model on communicative competence, this research…

The present study aimed to validate a 70-item Korean bilingual version of the Vocabulary Size Test (VST) using Rasch modeling. The goal was to assess the applicability of this Korean version of the VST for Korean learners of English in an English as a foreign language (EFL) context by examining validity evidence based on Messick's framework.…

This study presents the validation process of a listening test based on a communicative language test proposed by Bachman (Fundamental considerations in language testing, 1990). It was administered to third-grade high school students by the sixteen Korean Provincial Offices of Education for Curriculum and Evaluation in September 2012 to assess…

Despite the popularity of the TOEIC in the Korean society for over 30 years, few studies have investigated the understanding and usage of TOEIC scores in the Korean context. This research gap needs to be filled to provide test users with useful information in the Korean context. Using an argument-based approach to validation, this study…

The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The "TOEFL® Essentials"™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the…

The present study examines Korean high school English teachers' and university students' perceptions about the criterion-referenced assessment of English in CSAT. A total of 400 participants (149 teachers and 251 students) answered an online questionnaire. Teachers showed more positive attitudes than students in their perceptions about the…

This study is aimed at ensuring the validity of the Practical English Certification Test (PECT) of the Chung-nam Office of Education (COE) in Korea. Motivated by the demand for a developing localized English test to empower English learning in public education, the COE conducted the PECT for 38,544 students of elementary, middle and high schools…

A valid metacognitive awareness second language (L2) listening questionnaire is important for both researchers and teachers to facilitate the measurement of language learners' ability to reflect on and direct their L2 learning. However, very few metacognitive awareness questionnaires have been validated, particularly in the area of L2 listening…

This study examines the prevalence, contexts, and demographic correlates of monotonic response patterns (MRPs) in online student evaluations. Results of two-level hierarchical generalized linear models show evidence of careless monotonic responses to a survey administered to students enrolled in a university-level foreign language course in the…

Syntactic (i.e., grammatical) complexity refers to the range and the degree of sophistication of the forms that appear in language production (Ortega, 2003). This concept has long been regarded as an important construct of language proficiency and has been actively investigated in the field of second language (L2) writing. Syntactic complexity is…

This study aims to develop and validate the Scale of Self-Regulated Language Learning (S2RLL), intended to measure high school students' self-regulated language learning skills such as grit, self-efficacy, goal orientations, and language learning strategies. A total of 1281 Korean high school students participated in this study. Various evidence…

To investigate whether the type of keyboard used in exams introduces any construct-irrelevant variance to the TOEFL iBT Writing scores, we surveyed 17,040 TOEFL iBT examinees from 24 countries on their keyboard-related perceptions and preferences and analyzed the survey responses together with their test scores. Results suggest that controlling…

Scores on the Test of English for International Communication Listening and Reading (TOEIC LR) have been used for rolling-based university admissions in South Korea. Because this test is not designed for that purpose by the testing organization, there may be a need to explore the validity of TOEIC LR score use for admission purposes. Accordingly,…