In this paper we use historic score-reporting records and test-taker metadata to inform data-driven recommendations that support international students in their choice of undergraduate institutions for study in the United States. We investigate the use of Structural Topic Modeling (STM) as a context-aware, probabilistic recommendation method that…

As English continues to spread as an international lingua franca, there is a growing diversity in its use around the world. As a result, there are calls for embracing the diversity in the teaching, learning and assessment of the language. At the same time, there is a growing criticism against the widely taken language tests such as the Test of…

To account for natural variability in cognitive processing, it is standard practice to optimize a model's parameters by fitting it to behavioral data. Although most language-related theories acknowledge a large role for experience in language processing, variability reflecting that knowledge is usually ignored when evaluating a model's fit to…

Best linear prediction (BLP) and penalized best linear prediction (PBLP) are techniques for combining sources of information to produce task scores, section scores, and composite test scores. The report examines issues to consider in operational implementation of BLP and PBLP in testing programs administered by ETS [Educational Testing Service].

Background: Validity is a notable concept in language testing which has concerned many researchers and scholars in the field of language testing due to its importance in decision making process. Tests' results always introduce consequences to test takers' lives which emphasizes the need to ensure their validity. Detecting and delineating the…

In this research report, we describe the design and empirical findings for a large-scale study of essay writing ability with approximately 2,500 high school students in Germany and Switzerland on the basis of 2 tasks with 2 associated prompts, each from a standardized writing assessment whose scoring involved both human and automated components.…

Developing a diagnostic tool within the diagnostic measurement framework is the optimal approach to obtain multidimensional and classification-based feedback on examinees. However, end users may seek to obtain diagnostic feedback from existing item responses to assessments that have been designed under either the classical test theory or item…

This study investigates two models of L2 vocabulary acquisition at an advanced L2 level: L1 lemma mediation and the revised hierarchical models. Proponents of the L1 lemma mediation model postulate that advanced L2 learners map L2 words to L1 meanings or concepts, whereas proponents of the revised hierarchical model argue that the increasing…

In this study, we assessed the invariance in the factor structure underlying English-language proficiency for two groups of adolescent learners in Japan: students in middle school (ages 13-15 years) and students in high school (ages 16-18 years). Language proficiency was measured using the "TOEFL Junior"® Comprehensive test, an…

Although several studies have explored the effects of task sequencing on second language (L2) production, there is no established set of criteria to sequence tasks for learners in L2 writing classrooms. This study examined the effect of simple ?complex task sequencing manipulated along both resource-directing (± number of elements) and…

This study investigates 3 unexplored issues regarding Second Language (L2) Motivational Self System theory. It further validates the theory using multiple structural equation modeling (SEM) along with a procedure comparing the strength of corresponding paths. Japanese university freshmen (N = 2,631) responded to a questionnaire and took the…

The present study examined the reliability of the reading, listening, speaking, and writing section scores for the TOEFL iBT® test and their interrelationship in order to collect empirical evidence to support, respectively, the "generalization" inference and the "explanation" inference in the TOEFL iBT validity argument…

Despite calls for engaging assessments targeting young language learners (YLLs) between 8 and 13 years old, what makes assessment tasks engaging and how such task characteristics affect measurement quality have not been well studied empirically. Furthermore, there has been a dearth of validity research about technology-enhanced speaking tests for…

The present study examines the relative importance of attributes within and across items by applying four cognitive diagnostic assessment models. The current study utilizes the function of the models that can indicate inter-attribute relationships that reflect the response behaviors of examinees to analyze scored test-taker responses to four forms…

In the literature of second language teaching and learning, the use of students' first language (L1) has been an issue of debate for many years. Despite the changing state of teaching and learning aspects over the years, the belief that L1 should not be used in English as a Foreign Language (EFL) classrooms has stood the test of time.…