Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I…

Placement exams are high-stakes assessments that determine many students' college trajectories. More than half of entering students at community colleges are placed into developmental education in at least one subject, based primarily on scores from these assessments, yet recent research fails to find evidence that placement into remediation…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

In this commentary, the author addresses what is referred to as the deterministic input, noisy "and" gate (DINA) model. The author mentions concerns with how this model has been formulated and presented. In particular, the author points out that there is a lack of recognition of the confounding of profiles that generally arises and then discusses…

Rupp and Templin (2008) do a good job at describing the ever expanding landscape of Diagnostic Classification Models (DCM). In many ways, their review article clearly points to some of the questions that need to be answered before DCMs can become part of the psychometric practitioners toolkit. Apart from the issues mentioned in this article that…

In the summer of 2011, the Tennessee Department of Education contracted with the National Institute for Excellence in Teaching (NIET) to provide a four-day training for all evaluators across the state. NIET trained more than 5,000 evaluators intensively in the state model (districts using alternative instruments delivered their own training).…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

On a general level, the objective of diagnostic classifications models (DCMs) lies in a classification of individuals regarding multiple latent skills. In this article, the authors show that this objective can be achieved by multidimensional adaptive testing (MAT) as well. The authors discuss whether or not the restricted applicability of DCMs can…

Authentic assessment is a growing alternative to conventional testing. This research-to-practice article describes a framework for implementing authentic assessment. The R-E-A-L framework shows how roles, equipment, assessment tools, and location can be incorporated into early childhood practices.

The inclusion of young children with disabilities has remained a function of the Head Start program since its inception in the 1960s when the United States Congress mandated that children with disabilities comprise 10% of the Head Start enrollment (Zigler & Styfco, 2000). Standardized, norm-referenced tests used to identify children with…

In "Holding Accountability to Account: How Scholarship and Experience in Other Fields Inform Exploration of Performance Incentives in Education"--a paper presented at the National Center on Performance Incentives research to policy conference in February--Richard Rothstein, a research associate at the Economic Policy Institute, argues educational…

Mounting pressure in the policy arena to improve teacher productivity either by improving signals that predict teacher performance or through creating incentive contracts based on performance--has spurred two related questions: Are there important determinants of teacher productivity that are not captured by teacher credentials but that can be…

Comparison of student test scores between states, school districts, and even schools continues to be a popular measure of student achievement. However, these comparisons reveal little about the quality or effectiveness of educational programs, only the varying difficulty of educating different populations of students. This report uses U.S. Census…