This article is based on my 2023 NCME Presidential Address, where I talked a bit about my journey into the profession, and more substantively about comparable scores. Specifically, I discussed some of the different ways 'comparable scores' are defined, highlighted some areas I think we as a profession need to pay more attention to when considering…

The purpose of this ITEMS module is to provide an introduction to differential item functioning (DIF) analysis using mixture item response models. The mixture item response models for DIF analysis involve comparing item profiles across latent groups, instead of manifest groups. First, an overview of DIF analysis based on latent groups, called…

There is much discussion about and many policies to address achievement gaps in education among groups of students. The focus here is on a different gap and it is argued that it also should be of concern. Speed gaps are differences in how quickly different groups of students answer the questions on academic assessments. To investigate some speed…

Scores on essay-based assessments that are part of standardized admissions tests are typically given relatively little weight in admissions decisions compared to the weight given to scores from multiple-choice assessments. Evidence is presented to suggest that more weight should be given to these assessments. The reliability of the writing scores…

Some writers in the measurement literature have been skeptical of the meaningfulness of achievement standards and described the standard-setting process as blatantly arbitrary. We argue that standard setting is more appropriately conceived of as a measurement process similar to student assessment. The construct being measured is the panelists'…

Drawing on experience between 2000 and 2007 in developing a validity argument for the high-stakes Test of English as a "Foreign Language[TM]" (TOEFL[R]), this paper evaluates the differences between the argument-based approach to validity as presented by "Kane (2006)" and that described in the 1999 "AERA/APA/NCME Standards for Educational and…

This article describes an alignment study conducted to evaluate the alignment between Indiana's Kindergarten content standards and items on the Indiana Standards Tool for Alternate Reporting. Alignment is the extent to which standards and assessments are in agreement, working together to guide educators' efforts to support children's learning and…

Does it matter if students are appropriately assigned to test accommodations? Using a randomized method, this study found that individual students assigned accommodations keyed to their particular needs were significantly more efficacious for English language learners (ELLs) and that little difference was reported between students receiving…

This study compared different stakeholders' perceived validity of various indicators of student learning used to judge the quality of students' academic performance. Data were gathered from the questionnaire responses of 314 educators in three states that have implemented comprehensive state-wide assessment programs with high-stakes consequences…

This National Council on Measurement in Education (NCME) instructional module compares classical test theory and item response theory and describes their applications in test development. Related concepts, models, and methods are explored; and advantages and disadvantages of each framework are reviewed. (SLD)

Data from large-scale international studies for 13 countries indicate that U.S. students are clearly not the most heavily tested students on earth if one compares systemwide tests by their duration. In the United States, tests are much more likely to be of low consequence for the student. (SLD)

This instructional module discusses the one-, two-, and three-parameter logistic item response theory (IRT) models. Mathematical formulas are given for each model and they are compared, with figures illustrating the effects of changing parameters. A single data set is used to demonstrate the effects of changing parameter values. (SLD)

If annual performance assessments are to yield results that can be compared from year to year, many technical problems must be addressed. It is essential that tests to be equated measure the same construct. Methods of equating performance assessment scores, ways of equating system assessments, and standard setting are discussed. (SLD)

Proposed an accountably model that addresses the challenges of allowing school districts to choose the specific strategies they use to measure student performance and evaluated this model using data from multiple sources for all school districts in Florida. Findings identify three strategies that would be useful for this type of accountability…

Different methodologies for linking tests across languages are reviewed and evaluated, focusing on monolingual item response theory, bilingual group designs, and matched monolingual group designs. These methods, although not without weaknesses, are superior for promoting score comparability than methods that rely on translation or expert judgment…