Subscore reporting under item response theory models has always been a challenge partly because the test length of each subdomain is limited for precisely locating individuals on multiple continua. Diagnostic classification models (DCMs), providing a pass/fail decision and associated probability of pass on each subdomain, are promising…

The determination of Pass/Fail decisions over Borderline grades, (i.e., grades which do not clearly distinguish between the competent and incompetent examinees) has been an ongoing challenge for academic institutions. This study utilises the Objective Borderline Method (OBM) to determine examinee ability and item difficulty, and from that…

Decisions about progress through an academic programme are made by Boards of Examiners, on the basis of students' course assessments. For most students such pass/fail grading decisions are straightforward. However, for those students whose results are borderline (either at a pass/fail boundary or boundaries between grades) the exercise of some…

Over the past twenty years in performance testing a specific item type with distinguishing characteristics has arisen time and time again. It's been invented independently by dozens of test development teams. And yet this item type is not recognized in the research literature. This article is an invitation to investigate the item type, evaluate…

Scores are the most visible and widely used products of a testing program. The choice of score scale has implications for test specifications, equating, and test reliability and validity, as well as for test interpretation. At the same time, the score scale should be viewed as infrastructure likely to require repair at some point. In this report…

Standard-setting research has yielded a rich array of more than 50 standard-setting procedures, but practitioners are likely to be confused about which to use. By synthesizing the accumulated research on standard setting and progress monitoring, this study developed a three-dimensional taxonomy for conceptualizing and operationalizing the various…

Research studies that have investigated the use of mastery testing models and other related techniques are examined. A review of the procedures suggested for setting the pass fail point is also discussed. (Author/DEP)

Results suggest that different groups of judges do set similar examination standards when using the same procedure, and that the average of individual judgments does not differ significantly from group consensus judgments. Significant differences were found, however, between the standards set by the two procedures employed. (RC)

Data from two standard-setting exercises were analyzed using the logistic regression model that assumes no variation in severity of raters, and results were compared with those obtained by logistic regression that allowed for severity variation. Results illustrate the importance of taking between-rater differences into account. (SLD)

Standard setting on licensure and certification tests is difficult both to execute and to defend. There may, however, be certain minimum standards for standard setting on which most will be able to agree. One such standard is logical consistency. M. T. Kane (1984, 1986) has suggested an approach to evaluating the logical consistency of one widely…

A study was conducted to explore empirically the effect of different definitions of the target examinee on the judgment of panelists setting a passing score. Two cut score studies were done in a school district for the same test within a 6-month period, and different definitions of the target candidate were provided for each study. In October…

The objective of this study was to determine the degree to which recommendations for passing scores, calculated on the basis of a traditional standard-setting methodology, might be affected by the mode (paper versus computer-screen prints) in which test items were presented to standard setting panelists. Results were based on the judgments of 31…

The major forms of evidence that support a standard's credibility are reviewed, and what can be done over time and for different forms of an examination to enhance its comparability in a credentialing setting is outlined. Pass-fail decisions must be consistent to ensure a standard's credibility. (SLD)

Whether anchor item sets varying in difficulty and discrimination affect precision of cutting score equivalents generated through judge rescaling as much as equivalents from score equating was studied with 4 groups of experts and 250 and 1,000 examinees. Results indicate the robustness of judge rescaling and its superiority over equating. (SLD)

Reviews and evaluates compensatory and conjunctive standard-setting strategies, presenting and discussing the rationales for each approach. Results of a comparison of the two approaches for a state high school certification writing test provide insight into the problem of choosing either strategy. (SLD)