The assessment of differential item functioning (DIF) in polytomous items addresses between-group differences in measurement properties at the item level, but typically does not inform which score levels may be involved in the DIF effect. The framework of differential step functioning (DSF) addresses this issue by examining between-group…

It has often been asserted that the reliability of a measure places an upper limit on its validity. This article demonstrates in theory that validity can rise when reliability declines, even when validity evidence is a correlation with an acceptable criterion. Whether empirical examples can actually be found is an open question. (SLD)

The attempt by D.Edwards and C. B. Cummings to adjust state mean Scholastic Aptitude Test Scores for differential participation rates with a "fuzzy truncation model" satisfies three criteria the authors previously defined but falls short for two. Omission of sensitivity studies mars the otherwise exemplary study. (SLD)

When raters assign different scores to a performance task, a method for resolving rating differences is required to report a single score to the examinee. Recent studies indicate that decisions about examinees, such as pass/fail decisions, differ across resolution methods. Previous studies also investigated the interrater reliability of…

A brief summary of standard setting knowledge is presented, derived from about 20 methods that utilize a judgmental review process, the approach most relevant to the standard-setting strategies proposed in this special issue. Criteria for judging effectiveness and critiques of the methods discussed in the issue are offered. (SLD)

Using item response theory to investigate differential item functioning (DIF), students' expected course grades were examined and found to function similarly across sex and race. These grades were incorporated into the matching criterion, enhancing the validity of subgroup comparisons for the third-grade mathematics test taken by 1,050 students.…

This article provides a framework for the rest of the articles in this special issue comparing the utility of three standard-setting methods with complex performance assessments. The context of the standard setting study is described, and the methods are outlined. (SLD)