College and university admissions officers are the gatekeepers who decide which individuals may join the "country club" of higher education. As such, the decisions they make have a significant impact on the lives of individuals, especially those who are willing to travel across the globe and risk living in an unfamiliar country in order…

The authors observe in this commentary that construct maps can help standard-setting panels to make realistic and internally consistent recommendations for performance-level descriptions (PLDs) and cut-scores, but the benefits may not be realized if policymakers do not fully understand the rationale for the recommendations provided by the…

The construct map is a particularly good way to approach instrument development, and this author states that he was delighted to read Adam Wyse's thoughts about how to use construct maps for standard setting. For a number of popular standard-setting methods, Wyse shows how typical feedback to panelists fits within a construct map framework.…

The authors of this article comment on "Construct Maps as a Foundation for Standard Setting," by Adam E. Wyse (this issue) in which Wyse presents construct maps, a visual display of a variety of sources of evidence that support standard-setting decisions, and shows how this approach could be used with a variety of existing…

Research conducted by the Community College Research Center (CCRC) and others was criticized in an article by Alexandros M. Goudas and Hunter R. Boylan (2012) published in the "Journal of Developmental Education," Volume 36, Issue 1. They raise specific contentions related to the methodology applied in the CCRC studies, the review of…

A recent NBER [National Bureau of Economic Research] working paper examines Florida's policy to retain many low-scoring third graders. The report concludes that third-grade retention has immediate positive effects on the following year's test results, but these effects fade over the next six years, with no effect on graduation. The regression…

This monograph has grown out of a series of discussions and a six-month period of reading and reflecting on the literature which were initiated by Fritz Mosher's suggestions to the National Assessment of Educational Progress (NAEP) to examine the "standards" question. Conversations with Mosher himself and the staff of NAEP have been most…

On virtually all tests these days, there is a score that determines whether a student passes or fails, is proficient or not, or is being educated or left behind. This is the cut score. This author states that, while it might seem cynical, the fairest way to set the cutoff score is to decide in advance how many students a person wants to fail. This…

It is shown that the Angoff and bookmarking cut scores are examples of true score equating that in the real world must be applied to observed scores. In the context of defining minimal competency, the percentage "failed" by such methods is a function of the length of the measuring instrument. It is argued that this length is largely…

Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge…

The academic performance consequences of relying solely on non-cognitive factors for selecting applicants above a GPA and MCAT threshold have not been fully considered in the literature. This commentary considers the impact of using a "threshold approach" on academic performance as assessed with the USMLE Step 1.

"Jeopardy," the trivia game show first introduced in 1964 is approaching its 25th straight year on television. Next to "The Price is Right," it is arguably one of the greatest game shows of all time. Unfortunately, "Jeopardy!" is not just a game show. It is, according to this author, the game show gone reality show in…

This paper discusses how cut scores are set and used and how accurately they reflect student achievement. Regardless of the method used, the cut-score setting process is subjective. The cut score is the point on a score scale that separates one performance standard from another. Cut scores may also be used to set performance levels for…

Suggests a process for setting a cutoff score on knowledge tests for promotion, certification, and licensing: a modified Angoff method, in which a competency estimate is determined by subject matter experts, is combined with analysis of potential impact on any groups protected by the Civil Rights Act. (SK)

Some general questions about minimum competency tests are discussed, and various methods of setting standards are reviewed with major attention devoted to those methods used for dichotomizing a continuum. Methods reviewed under the heading of Absolute Judgments of Test Content include Nedelsky's, Angoff's, Ebel's, and Jaeger's. These methods are…