In large-scale educational assessment data consistent with a simple-structure multidimensional item response theory (MIRT) model, where every item measures only one latent variable, separate unidimensional item response theory (UIRT) models for each latent variable are often calibrated for practical reasons. While this approach can be valid for…

Covariational reasoning (or the coordination of two dynamically changing quantities) is central to secondary STEM subjects, but research has yet to fully explore its applicability to elementary and middle-grade levels within various STEM fields. To address this need, we selected a globally referenced STEM assessment--the Trends in International…

The chapters in this online volume comprise the TIMSS & PIRLS International Study Center's technical report of the methods and procedures used to develop, implement, and report the results of TIMSS 2019. There were various technical challenges because TIMSS 2019 was the initial phase of the transition to eTIMSS, with approximately half the…

Over time international large-scale assessments have grown in terms of number of studies, cycles, and participating countries, many of which are a heterogeneous mix of economies, languages, cultures, and geography. This heterogeneity has meaningful consequences for comparably measuring both achievement and non-achievement constructs, such as…

National Assessment of Educational Progress (NAEP) has expended considerable effort to ensure high quality in data collection by developing standardized materials and survey operation procedures and using well-trained professional administrators. However, schools are allowed to minimize the disruption associated with pulling students out of…

The assessment of differential item functioning (DIF) in polytomous items addresses between-group differences in measurement properties at the item level, but typically does not inform which score levels may be involved in the DIF effect. The framework of differential step functioning (DSF) addresses this issue by examining between-group…

This article introduces the Measurement Skills Assessment (MeSA), which was designed to evaluate the mastery of measurement in elementary school students. The primary objectives for the MeSA include covering a broad range of measurement concepts, distinguishing between major subtypes of measurement, and constructing a continuum of items varying in…

This paper describes the development and validation of the Cognate Awareness Test (CAT), which measures cognate awareness in Spanish-speaking English Language Learners (ELLs) in fourth and fifth grade. An investigation of differential performance on the two subtests of the CAT (cognates and noncognates) provides evidence that the instrument is…

In Spring 2006,. the NAEP Validity Studies (NVS) Panel was asked by the National Center for Education Statistics (NCES) to undertake a validity study to examine the quality of the NAEP Mathematics Assessments at grades 4 and 8. Specifically, NCES asked the NVS Panel to address five questions: (1) Does the NAEP framework offer reasonable content…

The purpose of this study was to document the instrument development of maze measures for grades 3-8. Each maze passage contained twelve omitted words that students filled in by choosing the best-fit word from among the provided options. In this technical report, we describe the process of creating, reviewing, and pilot testing the maze measures.…

In this technical report, we describe the development and piloting of a series of mathematics progress monitoring measures intended for use with students in grades kindergarten through eighth grade. These measures, available as part of easyCBM[TM], an online progress monitoring assessment system, were developed in 2007 and 2008 and administered to…

Universal design of assessment has been an important step forward in making tests more accessible to students with disabilities. An issue affecting the universal design approach is the need to review individual items, potentially hundreds of them. Ideally, there would be a statistical procedure available that would first identify items that are…

In the context of Phase Four of the National Assessment of Educational Progress (NAEP) Science Attribute Study, this report includes a discussion of item attributes, an overview of the item attributes used in the study, some psychometric characteristics of the blocks analyzed, a general description of the rule-space methodology, the results…

The impact of language characteristics in mathematics test items on student performance was evaluated for students with disabilities (SWD) and English language learners (ELL) as well as a large general student sample. Relationships were examined for test items and students at 4th, 7th, and 10th grades. The individual test item was the unit of…

The study described in this paper is part of an effort to improve understanding of the science assessment of the National Assessment of Educational Progress (NAEP). It involved the coding of all the items in the 1996 NAEP science assessments, which included 45 blocks (15 each for grades 4, 8, and 12) and over 500 items. Each of the approximately…