Virtual performance assessments (VPAs), such as game- and simulation-based assessments, provide promising ways for assessing complex and integrated skills. However, the high cost, long development cycle, and complex scoring process significantly hinder the adoption of VPAs, particularly in large-scale assessments with tight deadlines and limited…

This research report provides a description of the processes of evaluating the "deployability" of automated scoring (AS) systems from the perspective of large-scale educational assessments in operational settings. It discusses a comprehensive psychometric evaluation that entails analyses that take into consideration the specific purpose…

Assessment refers to a broad array of approaches for measuring or evaluating a person's (or group of persons') skills, behaviors, dispositions, or other attributes. Assessments range from standardized tests used in admissions, employee selection, licensure examinations, and domestic and international large-scale assessments of cognitive and…

In this study, we investigated the challenges and opportunities in developing a computer-delivered English language arts (ELA) task intended to improve the accessibility of the task for middle school English learners (ELs). Data from cognitive labs with 8 ELs with varying language proficiency levels provided rich insight to student-- task…

A common interpretation of achievement test results is that they provide measures of achievement that are much like other measures we commonly use for height, weight, or the cost of goods. In a limited sense, such interpretations are correct, but some nuances of these interpretations have important implications for the use of achievement test…

"Translanguaging" refers to the flexible use of the bilingual repertoire. In this report, we provide a theoretical framework to support the use of translanguaging to assess the academic content knowledge of newly arrived emergent bilingual students. In this report, we argue that translanguaging offers newly arrived emergent bilingual…

This project examined academic reading in two contrasting English as a medium of instruction (EMI) university settings in Nepal and Sweden and the unique challenges facing students who are studying in a language other than their primary language. The motivation for the project was to explore the role of high-stakes testing in EMI contexts and the…

Kindergarten entry assessments (KEAs) have increasingly been incorporated into state education policies over the past 5 years, with much of this interest stemming from Race to the Top--Early Learning Challenge (RTT-ELC) awards, Enhanced Assessment Grants, and nationwide efforts to develop common K-12 state learning standards. Drawing on…

The Mission Skills Assessment (MSA) is an online assessment that targets 6 noncognitive constructs: creativity, curiosity, ethics, resilience, teamwork, and time management. Each construct is measured by means of a student self-report scale, a student alternative scale (e.g., situational judgment test), and a teacher report scale. Use of the MSA…

Previous research on calculator use in standardized assessments of quantitative ability focused on the effect of calculator availability on item difficulty and on whether test developers can predict these effects. With the introduction of an on-screen calculator on the Quantitative Reasoning measure of the "GRE"® revised General Test, it…

The present study investigated the factor structure of a field trial sample of the Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT). An item-level confirmatory factor analysis (CFA) was conducted for a polychoric correlation matrix of items on a test form completed by 2,720 participants in the 2003-2004 TOEFL iBT Field…

The relatively new shadow test approach (STA) to computerized adaptive testing (CAT) proposed by Wim van der Linden is a potentially attractive alternative to the weighted deviation algorithm (WDA) implemented at ETS. However, it has not been evaluated under testing conditions representative of current ETS testing programs. Of interest was whether…

The purpose of this report is to present the findings of the first phase in a longitudinal study of the impact of changes in the TOEFL® test on teaching and learning in test preparation classrooms. Observations were carried out and interviews conducted with teachers, students, and directors of studies at 10 institutions in six countries in Central…