Background: Variations in the skill profiles of learners have become an important research area in recent years. However, there is a lack of empirical research on this topic in Japan. We conducted three studies to address this gap. Methods: Study 1 investigated the characteristics of the flat and uneven skill profiles of Japanese learners of…

To better understand the relationship between second-language (L2) listening comprehension and metacognitive awareness, we examined the moderating effects of listening tests and learner samples while focusing on aspects of metacognitive awareness. Students of English-as-a-foreign-language at a Japanese university (n = 75; the 2019 cohort) took the…

Despite the wide use of language tests as a basis for awarding English language course credits at Japanese universities, little has been published about how universities set policies on awarding credits according to external test scores. To narrow this gap, the characteristics of such policies were investigated in relation to the EIKEN Test in…

We addressed Deville and Chalhoub-Deville's (2006), Schoonen's (2012), and Xi and Mollaun's (2006) call for research into the contextual features that are considered related to person-by-task interactions in the framework of generalizability theory in two ways. First, we quantitatively synthesized the generalizability studies to determine the…

With the goal of enhancing the competitiveness of students in the workplace, nearly two-thirds of technical universities in Taiwan have adopted a variety of standardized English proficiency tests as exit requirements, including the widespread use of TOEIC. However, employers still name "insufficient English proficiency" as one of the…

Background: While numerous articles on Criterion® have been published and its validity evidence has accumulated, test users need to obtain relevant validity evidence for their local context and develop their own validity argument. This paper aims to provide validity evidence for the interpretation and use of Criterion® for assessing second…

Background: This study examined the factor structure of the Test of English for Academic Purposes (TEAP®) test--a recently developed academic English test measuring four skills among Japanese university applicants--and compared the structure to that of the Test of English as a Foreign Language Internet-based test (TOEFL iBT®), to investigate the…

Despite the importance of lexical diversity (LD) in L2 speaking and writing performance, LD assessment measures are known to be affected by the number of words analyzed in the text. This study aims to identify LD measures that are least affected by text length and can be used for the analysis of short L2 texts (50-200 tokens). We compared the…

This study examined the factor structure of the listening and reading sections of the revised Test of English for International Communication (TOEIC[R]) test. The data from the TOEIC IP (institutional program) test taken by 569 English learners were randomly split into two samples (n = 285 vs. 284). Four models (higher-order, correlated,…

Despite the recent increase of structural equation modeling (SEM) in language testing and learning research and Kunnan's (1998) call for the proper use of SEM to produce useful findings, there seem to be no reviews about how SEM is applied in these areas or about the extent to which the current application accords with appropriate practices. To…

A meta-analysis was conducted on the effects of multiple-choice and open-ended formats on L1 reading, L2 reading, and L2 listening test performance. Fifty-six data sources located in an extensive search of the literature were the basis for the estimates of the mean effect sizes of test format effects. The results using the mixed effects model of…