New Meridian Corporation has developed the "Quality Testing Standards and Criteria for Comparability Claims" (QTS). The goal of the QTS is to provide guidance to states that are interested in including content from the New Meridian item bank and intend to make comparability claims with "other assessments" that include New…

New Meridian Corporation has developed the "Quality Testing Standards and Criteria for Comparability Claims" (QTS). The goal of the QTS is to provide guidance to states that are interested in including content from the New Meridian item bank and intend to make comparability claims with "other assessments" that include New…

The purpose of this study was to develop a standard-setting method appropriate for use with a diagnostic assessment that produces profiles of student mastery rather than a single raw or scale score value. The condensed mastery profile method draws from established holistic standard-setting methods to use rounds of range finding and pinpointing to…

New Meridian Corporation has developed the "Quality Testing Standards and Criteria for Comparability Claims" (QTS) to provide guidance to states that are interested in including New Meridian content and would like to either keep reporting scores on the New Meridian Scale or use the New Meridian performance levels; that is, the state…

The author recalls that as a child, he grooved on teeter-totters. Also known as a seesaw, a teeter-totter is a long, narrow board that's elevated with a pivot point in the middle so that as one end goes down the other end goes up. When going up or going down, sometimes quite rapidly, teeter-totters can provide their two riders with some…

The 2018 Nebraska Student-Centered Assessment System (NSCAS) Summative technical report documents the processes and procedures implemented to support the Spring 2018 NSCAS Summative English Language Arts (ELA), Mathematics, and Science assessments by NWEA under the supervision of the Nebraska Department of Education (NDE). The technical report…

In this article, E. Matthew Schulz responds to Adam Wyse's article, "Construct Maps as a Foundation for Standard Setting." In doing so, he asserts that one of the most important ideas in Wyse's work is that information used in standard setting needs to be better represented through the use of graphics. However, he's not…

Construct maps are tools that display how the underlying achievement construct upon which one is trying to set cut-scores is related to other information used in the process of standard setting. This article reviews what construct maps are, uses construct maps to provide a conceptual framework to view commonly used standard-setting procedures (the…

Standard-setting studies utilizing procedures such as the Bookmark or Angoff methods are just one component of the complete standard-setting process. Decision makers ultimately must determine what they believe to be the most appropriate standard or cut score to use, employing the input of the standard-setting panelists as one piece of information…

The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA results are used to determine if children are either proficient in English or have English language skills in one of four pre-proficient categories (pre-emergent,…

Uses several thought experiments to explore the potential impact of the choice of criterion on the results of examinee-centered studies. Conclusions from these experiments are then used to examine the different cutting scores from several contrasting groups studies on the National Assessment of Educational Progress. (SLD)

In the traditional Angoff Standard Setting Method, experts are instructed to predict the possibility that a randomly selected, hypothetical minimally competent candidate will be able to answer each multiple choice question in the test correctly. These item performance estimates are averaged across panelists and aggregated to determine the minimum…

Problems inherent in setting standards/passing scores for criterion-referenced tests are discussed; and traditional methods of setting standards are reviewed. Three acceptable methods based on judgments of questions are discussed; their authors include, respectively: (1) W. H. Angoff (1971); (2) R. L. Ebel (1972); and (3) L. Nedelsky (1954). (SLD)

Discusses panels for standard setting and presents 10 guidelines for the selection of panel members for such studies. Panel members should themselves hold the license for which they are producing a cutting score, and they must be familiar with the requirements of the profession and the characteristics of the candidates. (SLD)

This paper reports the results of using several alternative methods of setting cut scores. The methods used were: (1) a variation of the Angoff method (1971); (2) a variation of the borderline group method; and (3) an advanced impact method (G. Dillon, 1996). The results discussed are from studies undertaken to set the cut scores for fourth grade…