Although there has been substantial work on argument-based approaches to validation as well as standard-setting methodologies, it might not always be clear how standard setting fits into argument-based validity. The purpose of this article is to address this lack in the literature, with a specific focus on topics related to argument-based…

Angoff-based standard setting is widely used, especially for high-stakes licensure assessments. Nonetheless, some critics have claimed that the judgment task is too cognitively complex for panelists, whereas others have explicitly challenged the consistency in (replicability of) standard-setting outcomes. Evidence of consistency in item judgments…

The authors observe in this commentary that construct maps can help standard-setting panels to make realistic and internally consistent recommendations for performance-level descriptions (PLDs) and cut-scores, but the benefits may not be realized if policymakers do not fully understand the rationale for the recommendations provided by the…

The Angoff method requires experts to view every item on the test and make a probability judgment. This can be time consuming when there are large numbers of items on the test. In this study, a G-theory framework was used to determine if a subset of items can be used to make generalizable cut-score recommendations. Angoff ratings (i.e.,…

Establishing cut scores using the Angoff method requires panelists to evaluate every item on a test and make a probability judgment. This can be time-consuming when there are large numbers of items on the test. Previous research using resampling studies suggest that it is possible to recommend stable Angoff-based cut score estimates using a…

This report documents a standard-setting study to determine recommended minimum scores (cut scores) needed on the Core and Advanced iSkills[TM] assessments for examinees to be considered at a foundational level of ICT literacy skill. Two foundational levels--one for each iSkills assessment--had been specified previously by the National ICT…